the full script and notes from my talk as well as the slides are available online. what i want to do in this blog post is focus on a key message from the talk. a good place to start is to watch this 45 second video clip from Disney's movie "UP."
"I have just met you and I love you."
in the clip, Dug is very happy to "meet" new visitors. lucky for this unusual dog, he can communicate with new visitors, too. that's because both parties share an understanding on how to communicate before they have even met the first time. that's how the Web was designed: to allow people who have not yet met to communicate their ideas and create links between each other. that's very powerful. it takes advantage of what is now known as the Power Law in linked systems.
another important feature of the Web is to allow machines to talk with each other before they have ever met. with my browser, i can follow links to other servers and, even though my browser has not ever "met" that server, my browser knows how to "talk" to that server. servers are not "islands" of information, they are part of a Web.
your API does not love me
sadly, the way most of us implement HTTP APIs does not support this fundamental aspect of the Web. instead, each API has it's own "language" and the only way to understand that language is to first read a great deal of human-readable documentation and then build a special client application for just that one single server. this means every API service is a "lonely island" that no one understands upon first meeting. a very sad thing, IMO.
another great thing about Dug the dog is that he is ready and willing to follow unexpected things he discovers. if he "sees" a squirrel, he is apt to simply stop what he is doing and go over the find out what is going on. Dug is ready for new experiences.
again, this is an important aspect of the Web. we can follow new links to new places and learn new things. Stu Charlton explains this important feature of Web-based interfaces using the phrase serendipitous reuse.
these are not the links you are looking for
but most Web APIs today want us to ignore new links. they lead developers to design purpose-build applications just for the links within a single service. they don't allow client applications to easily discover and use new links. often even adding links within the service (e.g. a new feature) results in a "new API" through the use of a versioning pattern. clients are told to ignore new things and that is not at all like the Web.
i want to love your API
we need to change the way we design and implement APIs on the Web. we need to get back to the fundamental features that make the Web so powerful. we need more "love" and more "squirrels."
describe the interface, not the service
one of the first things we can do is to start describing Web APIs as "abstract classes" with only an interface that explains what is possible. right now we use decription languages that provide all the implementation details for a single server. in OO-speak, we describe server instances. this leads to building those purpose-built clients i mentioned above.
instead, we could describe interfaces alone. the interface becomes the shared understanding. and we can do it in a way that others (both server and clients) can implement in a shared way. one possible solution for this is a format i am working on with Leonard Richardson and Mark Foster called ALPS. ALPS is meant to provide this "abstract class"-style description of web APIs. there are other possible ways to do this, too.
use hypermedia formats
another important way we can all help each other is to rely more on hypermedia formats like AtomPub HAL, Collection+JSON, Siren, UBER and others when implementing our APIs for the web. using these formats for APIs it the equivalent of using HTML for Web pages. they all offer a high degree of shared understanding "before we have even met" and that can increase the linking value of all APIs. using shared hypermedia formats makes is possible for a single client application to "talk" to multiple services without the need for special coding.
and that's what we all want, right? to increase shared understanding and to make it possible for both people and machines to say...
"I have just met you and I love you."
starting april 4th, i'll be on the road for close to two weeks. along the way i have the honor of bringing the API Academy message of developer-focused, enterprise-scale API design and implementation to the cites of Seoul, Tokyo, and Singapore. in each of these cities we'll be hosting a free 1/2 day seminar covering some of the most popular topics the API Academy offers in our private onsite training to companies the world over.
i will also have the chance to do some additional presentations and make new connections while on this trip. as much as i enjoy the workshops, it is the chance to connect with those i've only known online and to meet new people that really makes these trips a great experience.
WWW2014 in Seoul
while in Seoul, i have the honor of presenting a peer-reviewed paper to the WS-REST 2014 Workshop which is part of the World Wide Web Conference in Seoul. it is not often that i get the oppty to speak at events of this calibre and i am also looking forward to catching up with several people who work on W3C projects; people i rarely get to meet in person.
there will also be an informal meetup in Seoul on the evening of April 8th near the COEX complex where the WWW2014 event is held and not far from the API Academy public workshop on the 9th. i don't have all the details and promise to post them as soon as i have them.
RESTful Web APIs in Tokyo
i am very excited to announce that i will be attending a RESTful Meetup in Tokyo the evening of April 12th. this was organized, in part, by a group of people who have also been hosting a bi-weekly reading group for the book RESTful Web APIs.
this group popped up last year to allow people to come together and translate the English-language edition of "RESTful Web APIs" in 'real time' by taking turns reading the content and then discussing it as a group. Leonard Richardson and i are very grateful for this kind of enthusiasm and i am looking forward to meeting some of the people behind this col project.
i arrive in Singapore on monday, april 14th and don't have any additional meetups scheduled yet. if you're in Singapore and want to set up something, ping me and let's see if we can get something going while i am in town for the public workshop on the 15th.
ok, let's go!
the chance to visit customers, developers, and designers in Seoul, Tokyo, and Singapore really has me energized. if you've not yet signed up for one of the public workshops, please do. and come up and tell me 'hello.' i'd love to hear about what your working on and how the API Academy can learn from your experience and help you reach your goals for building great applications for the Web and the enterprtise.
This blog post covers material from my 2014-03-28 talk at #APIStrat in Amsterdam. feel free to check out the notes from my talk along with the slides and related videos online.
one of the challenges i think we face is the possibility of adding tens of billions of connected devices to the internet in the next few years. currently this Internet of Things is expected to grow from 20 billion connected devices in 2015 to 40 billion in 2020.
that's a lot of things
estimates vary widely, but most agree we have something less than 10 billion connected devices today including computers, handhelds, cars, and controller devices such as SCADA and others. so, adding tens of billions more in just a few short years is going to present some challenges.
a lot of little things
and, if you start to think about wearables, RFID, and micro/nano-size items, you realize that billions of these devices are going to be quite small. not just physically, but also capability and capacity-wise. these will not have the power or flexibility of a laptop or even a handheld mobile phone.
these new members of the "connected community" will be tiny, single-purpose, low-power devices that "do one thing and do it well." and it is likely that program-ability of these devices will be limited. you might be able to flash the memory or even tweak configurations, but a good number of these devices will not be hosts to custom code the way web servers and handhelds are today.
yet, we still need these devices to work together; even if only to publish device data, consume data from others devices and react to outside stimulus (lights on/off, heat, air movement, etc.). so how will we do that? how can we get things to interact w/ each other?
most developers today write "code" by stringing long lists of imperative statments together into a coherent set of instructions for machines. to most devlopers "programming" is just that - providing step-by-step instructions. this means we take on responsibility for any mishaps along the way whether they occur due to problems inherent in our instruction set or due to unexpected events outside our own instrution set such as other devices that do not respond to our requests or send us unexpected data, etc. doing this on tiny devices w/ limited capacity and ability is going to be a problem.
luckily, there is another way to "program" these devices.
rules, not code
we can also use a rules-based approach to programming. in this paradigm, programmers establish a set of coherent rules that tell the device how to respond to outside stimulus ("if the lights are on do X else do Y" or "if the motion detector reports true then turn on the light", etc.). and that is all that you do; you write simple rules and leave the device to it's own.... well, devices.
this means "programs" are much smaller (just a set of rules), easier to debug, and easier to safely modify. it also means the devices themselves don't need to do a great deal of work in order to "act" according to the rules.
that's what i mean by small. the code will be small, the code will be rules.
how does that help us w/ the IoT?
it seems unlikely that we'll be able to "program" billions of devices to safely and successfully interact with each other if we have to constanly provide step-by-step instructions on how each one operates and how they all inter-operate. instead we'll rely on simple devices that do one thing well and make the history actions available (as a stream of data, occasional dumps, etc.) to other devices authorized for access to this history.
it is also important to keep in mind that these devices will not be "phoning home" asking servers for guidance on how to proceed. the amount of traffic that tens of billions of new devices might generate if they need to constantly get instructions from distant server would be massive and a huge waste of time and bandwidth. instead, most IoT devices will act on thier own, making decisions in real time using the rules provided to them.
so, this is a big deal, right?
this way of thinking about how devices will work, how we will "program" them is a big change for many developers. but not all of them. there are people today who build "low-level" device handlers and other things that do pretty much what i describe here. motion detectors, security systems, etc. all operate on this model (simple rules executed in realtime). the difference we'll see in the near future is that more of us will need to start designing and coding for devices that use this rule-based model.
but yes, this is a big deal.
BTW - i see quite a number of people building "Internet of Things" apps and models still assuming that the devices will be high-powered fully-programmable computers typical of handhelds or laptops today. this is a mistake. while there may be some IoT devices with that profile, i suspect it will be a very small minority of the projected 40 billion devices by 2020.
so start thinking ahead. start planning for a new way to program devices. those who start working like this now will have a jump on their peers and competitors. and there is good reason to start contemplating this right away. rule-based systems will require different training and even different tooling. think about this will affect the "test-driven development" movement and related practices. and that's just one example.
are there examples out there of this "new" kind of programing?
yes, there are. and i'll pick up that thread in another blog post.
today i posted the first working draft of a new hypermedia design i am (humbly) calling "Uber Hypermedia." i've been sitting on a handful of media type designs for a couple years and finally have decided to turn my scraps of notes into something tangible. this gives me a chance to work out my ideas, get feedback from others, and -- in a small way -- contribute to the improvement of media type design in general.
here is some text from the introductory copy of this current draft spec:
The Uber message format is a minimal read/write hypermedia type designed to support simple state transfers and ad-hoc hypermedia-based transitions. This document describes both the XML and JSON variants of the format and provides guidelines for supporting Uber messages over the HTTP protocol.and
The Uber message model has a number of design goals:
- Keep the message structure as lean as possible.
- Support all the H-Factors in hypermedia controls.
- Be compatible with multiple protocols (e.g. HTTP, CoAP, etc.)
- Maintain fidelity for more than one base message format (XML, JSON, etc.)
"Less is More"
Ludwig Mies van der Rohe,
i've adopted the "Less is More" approach. this media-type design has only three elements:
uber (the doc root),
data (a recursive element for data and transitions), and
error (for when things go wrong).
my hope here is that keeping the model super simple will make is easier to implement parsers and validators as well as make it easier to convert internal object models into messages and back again.
and similar to Piet Mondrian and other members of the Dutch De Stijl movement, i wanted to focus on the minimalist, abstraction of what a hypermedia message is. to this end, i tried to reduce hypermedia down to the bare essentials of data and transition.
there is only one element (
data) to express both content (e.g. "Mike Amundsen") as well as
express all the possible hypermedia actions such as those contained in HTML's
repetition and recursion
i also find the
reptition and recursive patterns in the
minimal music of
Steve Reich (and others)
very appealing and have used that as an inspiration in creating a format where the real information is found
in continuously-nested copies of the same (
data) element, each with its own slight variation.
essentially, creating a service that supports the Uber Hypermedia format means embracing the recursion and repetition that altready exists in our computer models. hopefully, this approach will make it realtively easy to handle conversions from internal object trees into external Uber messages.
comment, clone, and pull
i've posted the working draft in github in hopes that those wishing to provide feedback can comment, add issues, clone and even submit pull requests to help me refine and improve the design. i have yet to build any client or server implementations yet (that's the next step) and would love to hear from those wishing to do the same.
so there it is; a hypermedia format with just three elements, eleven attributes, and only seven reserved strings. i'm a man of small ambitions for this project [grin]!
in a recent thread on the API-Craft news group a question came up about how one could go about selecting one of the more than one-half dozen new hypermedia types when it came time to implement your Hypermedia API. it's a pretty good thread since many of the designers of these new media types were in on the conversation.
i posted a couple ways to think about assessment of a media type's applicability to the target API and thought i'd elaborate on that a bit here. the first method i recommend is to inspect the hypermedia features of the media type itself. the second measure i recommend is to try to get some information about how the media type is used in the wild.
one way to inspect a media type design and see if it will work for your use case is to use the H-Factor list. this is a list nine properties of hypermedia that may (or may not) be supported in the media type design. i won't go into the details of H-Factors here (you can follow the link above for details). However, the list of factors offers a way to see which hypermedia actions such as navigation links, embedding links, query forms, write bodies, the ability to select post formats, etc. (all things that Web APIs - at the network level must deal with) are described and supported within the media type in question.
doesn't support idempotent updates (PUT and DELETE) but
Atom does. And while Atom
doesn't support describing adhoc queries (ala
And while Collection+JSON doesn't support describing adhoc network methods
(e.g. specifying POST, PUT, PATCH, etc.),
does. And so forth.
so one way you can decide which media types you want to support is to figure out which hypermedia features you need to support in your API and select from the list of media type designs that match those needs.
another way to think about selecting a media type design is to focus on the existing implementations "in the wild." this can give you a sense of the stability, usability, and flexibility of the design. here are three things to consider when assessing implementations:
- independent implementations
a design that has a high number of independent implementations can be a sign
that the design is easy to work with and that it has a broad reach/appeal.
it may also be a sign that the design itself is well understood and stable.
this measure is based on
David Clark's notion of "rough concensus and
running code" as a guide for standards review.
We reject: kings, presidents and voting.
We believe in: rough consensus and running code.
- inter-operable implementations
the number of inter-operable implementations is a worthwhile measure, too.
IOW, can two people who never met before each implement the same design and
have their applications work together? This can be an indication of what
Stu Charlton calls
some designs don't have this as a goal; instead they are aiming for optimizing
a custom interface between two "well-known" parties. one example of a design
that has "serendipitous reuse" as a goal is the
Maze+XML media type. there are
quite a number of successful implementations that all work together nicely.
The Web architecture is all about serendipity, and letting a thousand information flowers bloom, regardless of whether it serves some greater, over arching, aligned need.
- unexpected implementations
it's also good to consider whether the design results in any "unexpected uses"
- something that was not part of the original designers' goals. you'e probably
experienced something like this yourself. you're working with an app, a
coding framework, programming language, etc. and suddenly you notice something
totally novel and say "Wow! I didn't think was even possible with this!"
one of the early advocates for studying
refers to this feaure in the last line of his excellent 2min video on
The value of a well-designed object is when it has such a rich set of affordances that the people who use it can do things with it that the designer never imagined.
there are other ways to make judgments about the applicability of a media-type design for your use case. here are some that come to mind:
- user activity
- this was mentioned in the thread that sparked this blog post. it is a good idea to check out how much online activity there is regarding the media type. for example, the "Issues" list in github, any associated news groups or help forums, and - in the case of established open standards groups - lists hosted by the governing standards body (W3C, IETF, OASIS, etc.). a healthy online community is can be a sign of a robust design.
- influential authors
- sometimes the authors of a media type hold an important position in the online community. this alone can give the design an 'instant' broad reach that some other designs won't match. for example, the primary authors of the JSON API (Steve Klabnik and Yehuda Katz) are also the key contributors to a couple very popular programming tools (Ruby on Rails and Ember.js respectively). when they get invovled in a design, it is likely that many will see it and get a chance to work with it.
- influential users
- in some cases, a design will be picked up by big names in the industry and this can increase the reach of a design. for example the HAL media type was recently chosen by Amazon.com for their appStream API. Microsoft's OData specification is another example of a media type design that has the opportunity to touch quite a number of developers and projects. finally, adoption by "heavy hitters" in the industry may help in cases where you need to convince your team or management to adopt the design for your own use.
the choice is yours
there you have it: H-Factors, implementation in the wild, and community standing are just some of the ways in which you can begin to critically assess whether a particular media type design is right for your API implementation. the most important thing to keep in mind in all this is not which methods you use to decide on your media type choices. the most important thing to remember is that you actively assess and make a choice.
to that end, i'll add one more quote (from Roy Fielding) to close out this post:
[C]onsider how often we see software projects begin with adoption of the latest fad in architectural design, and only later discover whether or not the system requirements call for such an architecture. Design-by-buzzword is a common occurrence.