mca blog

my guess is you are familiar with the phrase: "If you have to ask, you can't afford it." that's not what i mean here. let me show you...

if they have to ask...

try this:

1. create a new Web API
2. implement it on some server - up and running
3. hand the a single URL to some client dev and say "there ya go!"

is the API self-descriptive? does it contain enough information in the responses to allow client devs to know what the API is for, what it is capable of, and how client devs can make valid requests to the server and properly parse the responses?

here are some questions for you...

1. how many assumptions do you have about your API?
2. are these assumptions shared by client devs?
3. all clients devs?
4. even ones who have never met you?

if the answer to any of those questions is "no" or "i am not sure" then it is likely that someone will need to ask you questions about how to properly use your API. that's no big deal, right?

...you didn't afford it

in everyday life, if people have to ask how to use a device (television remote, toaster, etc.) then you can be sure that device is "poorly afforded" - it's a weak design. we all know devices (esp. electronics) that come with huge manuals and complicated explanations - bummer.

in this respect, your API is the same as any other consumer device. it should be "well afforded." developers should not have to read the technical equivalent of Tolstoy's "War and Peace" before they can successfully use your API.

yes, you can supply detailed instructions in prose, provide a long list of possible methods, include lots of tables, etc. these are helpful for devs, but they can be daunting to read and cumbersome to maintain.

another approach is the include this kind of information in a machine-readable format. one that most devs will also understand quickly. a set of instructions that get automatically updated whenever your API changes - hypermedia controls in the response. why write a web page of documentation to tell devs to construct a URI and use that URI to execute an HTTP GET when you can just include that (and much more) in your API responses?

help your client devs out. throw 'em a bone, here. don't make them read pages of docs when you can just include the simple run-time instructions instead.

in conclusion

if they have to ask, you didn't afford it.

over the last few months i've been working w/ Leonard Richardson on a book. along the way we've been collecting lots of ideas, assessing how things have worked out over the years, etc.

one of the things that has come up several times -- so much that it will likely be covered in the book -- is the idea that some Web patterns are focused, not on representing the state of things on a server but actually designed to describe the things on a server. this may seem like a subtle, possibly meaningless difference. but as i go through various technologies, guidance, and implementation examples, i think this way of viewing efforts on the Web today (representing vs. describing) is important. and mixing them up can cause you unnecessary pain.

what is happening?

representing the state of things on a server is what most of us assume the Web is all about. "I'll just go to that Web page and see if my order has been shipped." or "Gee, I wonder where that movie I want to see is playing this weekend?" and so forth. we use the Web to see what is happening, what has happened, even with will happen at some future point.

most of the applications built for the Web fall into this same category. we want to "go to a place on the Web" and "get something done" or "start working on something" or "see if that work is completed" and so forth. those of us who work to design and implement applications on the Web spend a good deal of our time translating stuff that happens over time into something that can be expressed in 'resources' that 'client applications' can manipulate in order to achieve the desired results.

the technologies and techniques for this kind of work are very familiar to most of us. HTTP, HTML, and Javascript are at the "front lines" of this tech stack. IIS, Apache Server, ngnix and others are the server-side engines of this work and they, in turn rely on complied components and services that store, manipulate, and calculate data that passes up and down this stack. that is programming for the Web.

what is it about?

there is another view of the Web, too. one that focuses on what the Web "is about" -- what those 'resources' mean and what they can be used for. this is a kind of 'meta Web' that works to describe the very Web we all use when we want to get things done.

this meta Web allows people to ask questions such as "Hey, that movie that's playing at the local theater is about the Crimean War. Are there other sources on that subject I can access on the Web?" or "I just saw that my online order for canned herring is being shipped this week. Where can I go to find out more about the company that processed those pickled herring?" and so forth. the meta Web allows us to link things together in new ways - ways the original authors of some Web content didn't intend when they created their content. the meta Web makes that possible even without requiring co-ordination from these various content authors.

the technologies for creating and maintaining this meta data has been around almost as long as the tech stack for creating the data in the first place. the RDF family of formats ( RDF-XML, RDFa, turtle, etc.) are on the front lines of the meta Web. these are formats that are designed to make it easy to make 'assertions' about other resources on the Web; assertions that can even support a level of reasoning across the Web. there is a long history of formats and services built around this ability to describe what all the Web data "is about"

chocolate and peanut butter?

what has been interesting to me is that i'm seeing a blending of these two worlds (representing vs describing) in new (and sometimes confusing) ways. The JSON-LD and the Linked Data community have their roots firmly planted in the "describing" world. recently, however, i'm seeing evidence of using this "describing" tech to build solutions for the "representing" world. a recent example of this is the Hydra project. here is an attempt to combine the RDF-style data description with the state-driven feature of hypermedia. this is a very interesting project.

oil and water?

but this mixing of the two worlds is not without problems. one case that comes to mind is Drupal's foray into the hypermedia space. last year, they decided to add support for JSON-LD. however, after several months of sprints, the group decided to switch to supporting HAL - a message design rooted in the 'representing' world.

of course, one case does not make a trend. but i suspect more of these cases will crop up over the near term as people work to make their applications and services more fully "Of the Web." stepping into this space w/o full understanding of these two different worlds is likely to cause some pain, frustrate implementations, and bust schedules.

this stuff matters

so, what's that point in all this?

the differences between models designed to represent vs. models designed to describe matter.

as Leonard and i work through the new crop of media type designs and implementation patterns for the Web we're seeing both of these approaches on display. sometimes the model and the use case are in close alignment and sometimes there are notable mismatches between the technology and the intent of the implementors - and that's gonna hurt.

It's been just over a year since O’Reilly released my Building Hypermedia APIs with HTML5 and Node and more than two years since I first started work on the early drafts. Inspired by the important 2007 book RESTful Web Services" by Leonard Richardson and Sam Ruby, I wanted to produce something that would - like the Richardson/Ruby book - offer a clear mode of evaluation, establish shared understanding, and promote a meaningful dialog about designing and implementing solutions on the Web. I'd hoped for some level of success and expected a healthy amount of skepticism. What I didn't foresee was the encouragement and support I received from Leonard as he read my early drafts. He offered great advice on improving the book and, even though I wasn't able to implement all his suggestions, I know the book benefited greatly from his attention. As I did, too.

Now, I have the even greater privilege of working with Leonard and Sam on a new book - "RESTful Web APIs". It’s scheduled for completion by the end of Q1 2013 and should be available soon after. The preliminary discussions and early drafts are exciting and I am hard-pressed to keep up with the already rapid pace of development on the book.

Leonard has written an extensive post on both the background behind this new book and the current state of affairs vis-a-vis APIs, hypermedia, and the Web. I’ll leave that discussion in his capable hands. What I will say here is that this new book is not just a "2nd Edition" of Leonard and Sam’s "RESTful Web Services"; It's more of a "follow-up" seven years on. It will contain the same chronicling of the current challenges facing Web developers and architects as well as a collection of up-to-date guidance on designing and building solutions for the today’s Web.

I'm really looking forward to the work ahead and to feedback from those reading our early drafts and helping us test our code and our ideas. It’s going to be quite a project!

As interest in hypermedia-oriented implementations for the Web increases, I am getting more questions about a "proper" way to design and execute hypermedia designs. Along the way, I've developed a rather simple way to describe the contents of hypermedia messages and how to "think about message design" when creating an implementation for the Web.

It all boils down to three levels of information that all exist within the shared messages on the Web: Structure Semantics, Protocol Semantics, and Application Semantics. The good new is, all three of these are needed for any successful communication between client and server - no matter the implementation style. The bad news is, most of the time these levels are obscured or muddled at implementation time. For that reason many of us do not get the chance to think about them clearly. Hopefully this blog post will help in that department and spark some comment and additional thinking on the subject.

Structure Semantics

The first level is the Structure Semantics. This is the set of rules regarding how to create a well-formed message. XML has rather simple structure semantics. JSON's rules for well-formedness are a bit more vague but reachable since JSON.parse(...) turns out to be the ultimate arbiter of such things. Determining well-formedness of other, more complex media types (HTML, Atom, HAL, Collection+JSON) is tougher, but do-able even if external validators are not always available.

It turns out to be very difficult to build a successful Web implementation that does not contain structure semantics - and that's a good thing.

Protocol Semantics

The second level is the Protocol Semantics. This is the set of rules regarding how to fashion valid requests and responses using one or more message protocols. HTTP, FTP, WebSockets, etc. are all message protocols. Most implementations for the Web use human-readable documentation to describe these semantics. For example, typical RPC-style implementations document URIs, associated request rules (HTTP methods and headers), and any possible body content. Hypermedia-style implementations express the protocol semantics directly in the message (usually in the body, but sometimes in HTTP headers). The most common way this is done is to include rules for links and 'forms' that may appear in server responses (think HTML).

Registered media types don't often include hypermedia controls. For example, CSV, XML, and JSON have no definitions for hypermedia expression within the message. However some media types such as HTML, Atom, HAL, Collection+JSON (and others), do include the predefined ability to express protocol-level information within the message sent from the server.

Application Semantics

The third level is the Application Semantics. This is the set of rules that govern the way in which application-specific information is shared between client and server. This includes names of domain data elements (users, customers, products, etc.) and specific actions within the domain (adding a user, calculating tax on purchases, etc.). Well-formed requests and responses are ones that contain valid expressions of these domain-specific concepts. There are very few registered media-type designs that contain this level of specificity and the ones that do contain it are usually are aimed at an industry-wide domain (e.g. VoiceXML and Maze+XML).

There are some machine-readable examples of application semantics that can be used in conjunction with existing structurally semantic forms (XML, JSON, etc.). XSD, JSON-Schema and other schema-based document types do a good job of expressing application-specific information. There are other example such as WSDL for SOAP and it's cousin WADL.

Most of the application-level semantics exist as stand-alone content with no standardized way to apply that information to a message. Some media type designs are better than others at supporting application-level semantics. HTML, for example, as several attributes (id, name, class, rel) with which you can apply application-level semantic information to a message.

Design Hallmarks

When designing a message format, you will always need to solve for all three levels (structure, protocol, and application). The simplest approach is to select an existing structural format (XML, JSON, etc.) and then use human-readable documentation to apply the other two (protocol and application). Doing this puts an added implementation coordination between client and server, but is quite common.

Selecting (or designing) media types that include protocol semantics and/or support applying application semantics directly within the message puts added responsibility on the message designer(s) and standardizes much more of the way client and servers interact. This can increase the likelihood that compatible implementations can be done at a distance (both in space and time) and that implementors (client and server developers) have an opportunity to safely evolve communications over time including adding new interactions to the domain-space without the need for prior coordination with the initial message designer.. It is these two aspects of the Web (support for distance and evolvability) that are a hallmark of hypermedia-style designs.

i'm happy to annnounce the release of the first "API Academy" video shorts. i've been working with my colleague, Ronnie Mitra, to create a series of short (5 min) informative videos on topics related to the Web, APIs, and solution design/implementation.

looking for feedback

these first few vids are just the start. over the coming weeks and months we plan on doing more of these shorts on a wide range of topics. and we need your help. please check out these first vids and send us your feedback. You can comment here, on the YouTube site, or email me directly. we're looking for feedback on the format, on suggested topics, and even on how we could improve on this model (hosting a separate site, adding interaction, badges, etc.).

thanks for your help

any time you can spend on these and sending comments will be most appreciated. out aim is to do something helpful, engaging, and - above all - enjoyable. thanks for your help and let's see what this can become!