this week i had the opportunity to deliver a "lightning talk" at the APIStrat Tech Un-Workshop at Gluecon 2014 . the event was focused on two key topics: IoT and Service Description and Discovery. i was in the Service Description/Discovery track and delivered a talk called "Mapping the API Landscape" (slides). i won't cover the entire talk here (the text BTW has lots of links to information i could not discuss on stage this week) but did want to hit some key points.
what Google's self-driving car tells us
the "gCar" has been in the news again and a key point that was disussed at some length in these articles was the fact that the car depends on a very detailed map of the roadways. in fact, the car currently can only drive in the Mountain View, CA area since that is the only landscape that is mapped well enough for the car to navigate.
so, the Google car does not "discover" anything while drivig. it actually recognizes intersections, traffic lights, etc. through a special representation of the landscape that contains all the right annotations. this reliance on a known map allows the car to navigate successfully between two points within that landscape. this is no simple feat, of course. reacting to surroundings "at speed" takes serious computing power and that's one of the things that makes the Google implementation so amazing.
Norman's Action Lifecycle
the process of navigating from A to B is a goal-driven process that we see very often in nature. ants, micro-organisms, etc. all do this. HCI expert, Donald Norman calls this process the Action Lifecycle or Seven Stages of Action .
this is how we learned to write GUI intefaces, too. wait for a keystroke or button-click, process that action, affect the UI, then allow the user to evaluate the changes and decide if another action is needed. we build Web servers like this, too. wait for a request, process it, modify the back-end (if needed) and reflect results back to the requestor. game programming works like this, too. but it's rare to see "Web Clients" written this way. they continue to look like single-minded bots that just "go from A to B" and ignore landmarks; incapable of actually reacting to surroundings.
client apps and web maps
why are most web client apps "one-off" implementations that are likely to break over time? because client dev's don't have decent "maps" of the Web landscape. and good maps are not just "photos" of the surroundings, but heavily annotated representations with recognizable symbols and landmarks. most servers today just belch out JSON or XML with almost no recognizable symbols or signage (e.g. hypermedia controls, etc.).
so what we need to do is create maps for devs to use so that they can build their own client applications and solve their own problems. client apps should be free to follow whatever route within that map that they wish -- not just follow the path that server developers decide upon.
let's make maps!
things like Service Description formats, and discovery protocols are all ways to start creating more maps for client devs to rely upon. using hypermedia in responses provides the symbols and signs client apps can use at runtime (like the Google car) in order to navigate in realtime.
there are several description formats (see my paper Hold your Nose vs. Follow your Nose for more on this). in the book, RESTful Web APIs Leonard Richardson and i list close to 20 options for expressing hypermedia in Web responses. and more have come online since the book was published last year.
we have all we need. we just need to make more maps!
the full script and notes from my talk as well as the slides are available online. what i want to do in this blog post is focus on a key message from the talk. a good place to start is to watch this 45 second video clip from Disney's movie "UP."
"I have just met you and I love you."
in the clip, Dug is very happy to "meet" new visitors. lucky for this unusual dog, he can communicate with new visitors, too. that's because both parties share an understanding on how to communicate before they have even met the first time. that's how the Web was designed: to allow people who have not yet met to communicate their ideas and create links between each other. that's very powerful. it takes advantage of what is now known as the Power Law in linked systems.
another important feature of the Web is to allow machines to talk with each other before they have ever met. with my browser, i can follow links to other servers and, even though my browser has not ever "met" that server, my browser knows how to "talk" to that server. servers are not "islands" of information, they are part of a Web.
your API does not love me
sadly, the way most of us implement HTTP APIs does not support this fundamental aspect of the Web. instead, each API has it's own "language" and the only way to understand that language is to first read a great deal of human-readable documentation and then build a special client application for just that one single server. this means every API service is a "lonely island" that no one understands upon first meeting. a very sad thing, IMO.
another great thing about Dug the dog is that he is ready and willing to follow unexpected things he discovers. if he "sees" a squirrel, he is apt to simply stop what he is doing and go over the find out what is going on. Dug is ready for new experiences.
again, this is an important aspect of the Web. we can follow new links to new places and learn new things. Stu Charlton explains this important feature of Web-based interfaces using the phrase serendipitous reuse.
these are not the links you are looking for
but most Web APIs today want us to ignore new links. they lead developers to design purpose-build applications just for the links within a single service. they don't allow client applications to easily discover and use new links. often even adding links within the service (e.g. a new feature) results in a "new API" through the use of a versioning pattern. clients are told to ignore new things and that is not at all like the Web.
i want to love your API
we need to change the way we design and implement APIs on the Web. we need to get back to the fundamental features that make the Web so powerful. we need more "love" and more "squirrels."
describe the interface, not the service
one of the first things we can do is to start describing Web APIs as "abstract classes" with only an interface that explains what is possible. right now we use decription languages that provide all the implementation details for a single server. in OO-speak, we describe server instances. this leads to building those purpose-built clients i mentioned above.
instead, we could describe interfaces alone. the interface becomes the shared understanding. and we can do it in a way that others (both server and clients) can implement in a shared way. one possible solution for this is a format i am working on with Leonard Richardson and Mark Foster called ALPS. ALPS is meant to provide this "abstract class"-style description of web APIs. there are other possible ways to do this, too.
use hypermedia formats
another important way we can all help each other is to rely more on hypermedia formats like AtomPub HAL, Collection+JSON, Siren, UBER and others when implementing our APIs for the web. using these formats for APIs it the equivalent of using HTML for Web pages. they all offer a high degree of shared understanding "before we have even met" and that can increase the linking value of all APIs. using shared hypermedia formats makes is possible for a single client application to "talk" to multiple services without the need for special coding.
and that's what we all want, right? to increase shared understanding and to make it possible for both people and machines to say...
"I have just met you and I love you."
starting april 4th, i'll be on the road for close to two weeks. along the way i have the honor of bringing the API Academy message of developer-focused, enterprise-scale API design and implementation to the cites of Seoul, Tokyo, and Singapore. in each of these cities we'll be hosting a free 1/2 day seminar covering some of the most popular topics the API Academy offers in our private onsite training to companies the world over.
i will also have the chance to do some additional presentations and make new connections while on this trip. as much as i enjoy the workshops, it is the chance to connect with those i've only known online and to meet new people that really makes these trips a great experience.
WWW2014 in Seoul
while in Seoul, i have the honor of presenting a peer-reviewed paper to the WS-REST 2014 Workshop which is part of the World Wide Web Conference in Seoul. it is not often that i get the oppty to speak at events of this calibre and i am also looking forward to catching up with several people who work on W3C projects; people i rarely get to meet in person.
there will also be an informal meetup in Seoul on the evening of April 8th near the COEX complex where the WWW2014 event is held and not far from the API Academy public workshop on the 9th. i don't have all the details and promise to post them as soon as i have them.
RESTful Web APIs in Tokyo
i am very excited to announce that i will be attending a RESTful Meetup in Tokyo the evening of April 12th. this was organized, in part, by a group of people who have also been hosting a bi-weekly reading group for the book RESTful Web APIs.
this group popped up last year to allow people to come together and translate the English-language edition of "RESTful Web APIs" in 'real time' by taking turns reading the content and then discussing it as a group. Leonard Richardson and i are very grateful for this kind of enthusiasm and i am looking forward to meeting some of the people behind this col project.
i arrive in Singapore on monday, april 14th and don't have any additional meetups scheduled yet. if you're in Singapore and want to set up something, ping me and let's see if we can get something going while i am in town for the public workshop on the 15th.
ok, let's go!
the chance to visit customers, developers, and designers in Seoul, Tokyo, and Singapore really has me energized. if you've not yet signed up for one of the public workshops, please do. and come up and tell me 'hello.' i'd love to hear about what your working on and how the API Academy can learn from your experience and help you reach your goals for building great applications for the Web and the enterprtise.
This blog post covers material from my 2014-03-28 talk at #APIStrat in Amsterdam. feel free to check out the notes from my talk along with the slides and related videos online.
one of the challenges i think we face is the possibility of adding tens of billions of connected devices to the internet in the next few years. currently this Internet of Things is expected to grow from 20 billion connected devices in 2015 to 40 billion in 2020.
that's a lot of things
estimates vary widely, but most agree we have something less than 10 billion connected devices today including computers, handhelds, cars, and controller devices such as SCADA and others. so, adding tens of billions more in just a few short years is going to present some challenges.
a lot of little things
and, if you start to think about wearables, RFID, and micro/nano-size items, you realize that billions of these devices are going to be quite small. not just physically, but also capability and capacity-wise. these will not have the power or flexibility of a laptop or even a handheld mobile phone.
these new members of the "connected community" will be tiny, single-purpose, low-power devices that "do one thing and do it well." and it is likely that program-ability of these devices will be limited. you might be able to flash the memory or even tweak configurations, but a good number of these devices will not be hosts to custom code the way web servers and handhelds are today.
yet, we still need these devices to work together; even if only to publish device data, consume data from others devices and react to outside stimulus (lights on/off, heat, air movement, etc.). so how will we do that? how can we get things to interact w/ each other?
most developers today write "code" by stringing long lists of imperative statments together into a coherent set of instructions for machines. to most devlopers "programming" is just that - providing step-by-step instructions. this means we take on responsibility for any mishaps along the way whether they occur due to problems inherent in our instruction set or due to unexpected events outside our own instrution set such as other devices that do not respond to our requests or send us unexpected data, etc. doing this on tiny devices w/ limited capacity and ability is going to be a problem.
luckily, there is another way to "program" these devices.
rules, not code
we can also use a rules-based approach to programming. in this paradigm, programmers establish a set of coherent rules that tell the device how to respond to outside stimulus ("if the lights are on do X else do Y" or "if the motion detector reports true then turn on the light", etc.). and that is all that you do; you write simple rules and leave the device to it's own.... well, devices.
this means "programs" are much smaller (just a set of rules), easier to debug, and easier to safely modify. it also means the devices themselves don't need to do a great deal of work in order to "act" according to the rules.
that's what i mean by small. the code will be small, the code will be rules.
how does that help us w/ the IoT?
it seems unlikely that we'll be able to "program" billions of devices to safely and successfully interact with each other if we have to constanly provide step-by-step instructions on how each one operates and how they all inter-operate. instead we'll rely on simple devices that do one thing well and make the history actions available (as a stream of data, occasional dumps, etc.) to other devices authorized for access to this history.
it is also important to keep in mind that these devices will not be "phoning home" asking servers for guidance on how to proceed. the amount of traffic that tens of billions of new devices might generate if they need to constantly get instructions from distant server would be massive and a huge waste of time and bandwidth. instead, most IoT devices will act on thier own, making decisions in real time using the rules provided to them.
so, this is a big deal, right?
this way of thinking about how devices will work, how we will "program" them is a big change for many developers. but not all of them. there are people today who build "low-level" device handlers and other things that do pretty much what i describe here. motion detectors, security systems, etc. all operate on this model (simple rules executed in realtime). the difference we'll see in the near future is that more of us will need to start designing and coding for devices that use this rule-based model.
but yes, this is a big deal.
BTW - i see quite a number of people building "Internet of Things" apps and models still assuming that the devices will be high-powered fully-programmable computers typical of handhelds or laptops today. this is a mistake. while there may be some IoT devices with that profile, i suspect it will be a very small minority of the projected 40 billion devices by 2020.
so start thinking ahead. start planning for a new way to program devices. those who start working like this now will have a jump on their peers and competitors. and there is good reason to start contemplating this right away. rule-based systems will require different training and even different tooling. think about this will affect the "test-driven development" movement and related practices. and that's just one example.
are there examples out there of this "new" kind of programing?
yes, there are. and i'll pick up that thread in another blog post.
today i posted the first working draft of a new hypermedia design i am (humbly) calling "Uber Hypermedia." i've been sitting on a handful of media type designs for a couple years and finally have decided to turn my scraps of notes into something tangible. this gives me a chance to work out my ideas, get feedback from others, and -- in a small way -- contribute to the improvement of media type design in general.
here is some text from the introductory copy of this current draft spec:
The Uber message format is a minimal read/write hypermedia type designed to support simple state transfers and ad-hoc hypermedia-based transitions. This document describes both the XML and JSON variants of the format and provides guidelines for supporting Uber messages over the HTTP protocol.and
The Uber message model has a number of design goals:
- Keep the message structure as lean as possible.
- Support all the H-Factors in hypermedia controls.
- Be compatible with multiple protocols (e.g. HTTP, CoAP, etc.)
- Maintain fidelity for more than one base message format (XML, JSON, etc.)
"Less is More"
Ludwig Mies van der Rohe,
i've adopted the "Less is More" approach. this media-type design has only three elements:
uber (the doc root),
data (a recursive element for data and transitions), and
error (for when things go wrong).
my hope here is that keeping the model super simple will make is easier to implement parsers and validators as well as make it easier to convert internal object models into messages and back again.
and similar to Piet Mondrian and other members of the Dutch De Stijl movement, i wanted to focus on the minimalist, abstraction of what a hypermedia message is. to this end, i tried to reduce hypermedia down to the bare essentials of data and transition.
there is only one element (
data) to express both content (e.g. "Mike Amundsen") as well as
express all the possible hypermedia actions such as those contained in HTML's
repetition and recursion
i also find the
reptition and recursive patterns in the
minimal music of
Steve Reich (and others)
very appealing and have used that as an inspiration in creating a format where the real information is found
in continuously-nested copies of the same (
data) element, each with its own slight variation.
essentially, creating a service that supports the Uber Hypermedia format means embracing the recursion and repetition that altready exists in our computer models. hopefully, this approach will make it realtively easy to handle conversions from internal object trees into external Uber messages.
comment, clone, and pull
i've posted the working draft in github in hopes that those wishing to provide feedback can comment, add issues, clone and even submit pull requests to help me refine and improve the design. i have yet to build any client or server implementations yet (that's the next step) and would love to hear from those wishing to do the same.
so there it is; a hypermedia format with just three elements, eleven attributes, and only seven reserved strings. i'm a man of small ambitions for this project [grin]!