RESTful Exploration

HATEOAS as Pictures

2012-02-10

HATEOAS as State Machines

HATEOAS (Hypermedia As The Engine Of Application State) is an essential part of REST. The application state part leads some to explain REST and HATEOAS using state machines. While this is technically correct, I really don’t like the idea of explaining REST as a state machine. People’s eyes glaze over immediately when you merely say state machine. The argument roughly goes like this:

1) If I make real applications with REST, then I must use state machines
2) I have made real applications without using state machines
3) Therefore I can make real applications without REST

State machines seem to be looked at as impractical by many for design purposes. While I don’t know if I agree with this statement, I think there is a need to explain things to people who hold the above argument.

HATEOAS as Pictures

Explaining HATEOAS as pictures seems more helpful to me. Imagine taking a picture of a small college administrative office. In the office there is a bunch of applications from students on the desk in the inbound folder. Right next to the inbound folder there are two folders labeled outbound-accepted and outbound-rejected. Also on the desk is an official course listing along with a rolodex labeled “Professors”.

In that picture you have a significant amount of information about the college. There is a small problem here though. What can trigger the adding of a new application into an inbound or outbound folder? That might be someone bringing in an application letter. What is it that triggers a new rolodex entry? Most likely a new professor joining the staff. Even though we can envision these actions, it’s hard to see these events in pictures.

HATEOAS events

One simple thing you can do is ask yourself “What all could cause a change in this picture?”, and list what you find. Notice I said what can cause a change, not what changes. You already have everything that actually changes explicitly in the picture.

Once you have a list of causes (or you could call these ‘events’ or ‘transitions’) and a list of entities (you can call these ‘resources’) in the picture, you can start making a true RESTful API. For every entity (resource) you’ll have a bunch of causes (transitions) that apply to it. You can arrange these like so:

Event
		New Application	New Professor	Accept Letter	Reject Letter
Entity	Inbound Box	Inbound Increased		Inbound Decreased	Inbound Decreased
	Outbound Box - Accepted			Outbound - Accepted Increased
	Outbound Box - Rejected				Outbound - Rejected Increased
	Rolodex		Rolodex Increased

Each column of that table is a picture. For instance, after a new application event happens, the inbound box will have a certain look, so will the outbound boxes, and even the rolodex. This is the case even though nothing but the inbound box can be affected by the new application event. Those other entities continue to exist so this still makes for a coherent picture.

All you have to do from here is transcribe your pictures into html and http. How do we do this? Well here is a representation of an inbox with the “new application”, “Accept Application”, and “Reject Application” events embedded inside of it:

<!-- Representing a list of college applications -->
<div id="Inbox">
  <a rel="new-application" href="..." 
      title="Create new application">New Application</a>
  <ul class="all">
    <li>
      <span class="student-application">John Smith's application</span>
      <a rel="accept" href="..." title="Reject John's application">Accept</a>
      <a rel="reject" href="..." title="Accept John's application">Reject</a>
    </li>
    <li>
      <span class="student-application">Jane Smith's application</span>
      <a rel="accept" href="..." title="Reject Jane Smith's application">Accept</a>
      <a rel="reject" href="..." title="Accept Jane Smith's application">Reject</a>
    </li>
  </ul>
</div>

The above html represents the idea of a list of applications (the inbox), and what you can do with that list (the events affecting the inbox). In this example you have the ability to add to the list (new application), or decrease the list (accepting or rejecting a list moves the application out of the inbox and into an outbox). The entities translate into html blocks, while the events translate into link relations.

What does all of this mean?

It means you can design applications in many ways according to what you need. It also means that there is no need to write REST off because it has a constraint that requires the use of state machines.

REST and HTTP Part 6

2011-10-24

If you’ve been following along so far, you’ll note that REST is not HTTP. It is also not the verbs of HTTP (GET, PUT, POST, or DELETE). You should know by now that REST contains the principles that guided HTTP. REST helped the HTTP standard describe how a well defined web server should behave.

The Fielding dissertation highlights how REST relates to HTTP mostly by way of its mismatches with HTTP. REST was used to identify problems and to compare solutions to those problems according to the requirements of the web. The early HTTP implementations were changed to more directly adhere to the REST architecture represented in the final HTTP 1.1 standard.

Many of the problems with early HTTP outlined in the REST paper were due to a lack of self describing messages. If you recall from our previous discussion, the requirement for messages to be self describing is one of the interface constraints within REST.

There were several mismatches of REST with the early implementations of HTTP. Early http lacked full URLs (with hostname), had no way to layer MIME types, and had no way to signal the end of a message. It also conflated the parsing of headers with the parsing of header contents, had no content negotiation, could not differentiate authoritative from non-authoritative responses, and in some situations had problems with matching responses to requests altogether. These were for the most part corrected in HTTP 1.1 which was standardized in 1996.

In HTTP 1.1 however, there are still some mismatches. URIs and cookies can and are abused to the point of obscuring messages. Mandatory extensions for headers violate self descriptiveness as well. Sometimes control data and other metadata can be intermingled. MIME is also inefficient when describing the media-type because it was designed for mail not hypermedia.

Nevertheless, REST helped HTTP become the primary protocol for web component communication and is unique in that it is designed specifically to manipulate resources via representation transfer. REST redefined what a resource meant within the context of HTTP. Sending representations, usually implemented in HTML, became the way to manipulate the resource on the remote server. REST also helped HTTP avoid extensions that would limit HTTP’s scalability. Some of these extensions where MGET/MHEAD, write (PUT) caching, various persistent connection solutions, and some http versioning solutions. At the same time it helped guide other unimplemented solutions like using self describing messages to implement HTTP’s versioning scheme and the implementation of caching.

REST Constraints Part 5

2011-10-13

Previously, we talked about the role of properties when evaluating architectures for network-based applications. We also talked about the paper’s working definition of architectural elements such as components along with the requirements of the web. Now we will use that knowledge to understand the paper’s definition of REST architecture as a set of constraints.

According to the REST dissertation, a constraint is a rule that induces one or more software architecture properties. A group of constraints is called a style. The REST style is a group of six major constraints that induce the various properties needed for the Web. In order for an architecture to be deemed a REST architecture, it must satisfy all of these constraints.

The first major constraint is the client-server constraint. The client-server constraint is based on a principle in software known as the separation of concerns. It simply requires the existence of a client component that sends requests and a server component that receives requests. After receiving a request the server may also issue a response to the client. An example of the client-server constraint is the separation of the user interface of a banking application into a client from the request-handling logic which would reside on the bank’s servers. The properties induced by the client-server constraint are portability, evolvability, and scalability.

The layered system constraint defines a REST architecture as hierarchical layers of components, limited to communication with their immediate neighbors. Similar to the client-server constraint, this constraint improves simplicity by separating concerns. It also improves extensibility by using layers as a mechanism to isolate legacy components. Scalability is increased by providing the capacity for intermediary components that act as caches or load balancers. The trade off with this constraint is reduced performance from the increased levels of indirection associated with multiple layers.

The stateless communication constraint requires every request to contain all of the information needed to interpret that request. This increases visibility, reliability, scalability, but also decreases performance because of the larger messages required for stateless communication.

The caching constraint states that a REST architecture must have some mechanism for labeling messages as cacheable or non-cacheable. This gives a client or intermediary component permission to cache and reissue a message which increases efficiency, scalability, and performance.

The code on demand constraint is an optional constraint. Code on demand allows a client to download and execute code from a server. This allows the client functionality to be extended after initial deployment of the application. This has an effect on simplicity as well, because it promotes the reduced coupling of features. Scalability is also increased by virtue of the server off-loading work onto the clients. The trade off is the reduced visibility generated by the code itself, which is hard for an intermediary to interpret. With this constraint being optional, the preferences of different organizations are served by allowing them the flexibility to deploy only when the trade-offs of deployment are worth it.

At a high level the uniform interface constraint basically means the interface for a component needs to be generic as possible. This increases evolvability, simplicity, and visibility but has the side effect of reducing efficiency given the nature of generic data. More specifically, the uniform interface constraint is defined by four interface sub-constraints, each of which warrant their own discussion.

Resources and Representations

In order to understand the interface sub-constraints, it is necessary to understand the idea of a resource. The dissertation says a resource is any information that can be named. It also says a resource is a conceptual mapping to a set of entities. In both cases a resource is always a concept, never a concrete thing. An example of a resource is the current weather in Austin, TX.

A representation is a sequence of bytes and the description of those bytes. Less formally, a representation is one form of a resource. An example of a representation is an html page with an embedded doppler radar PNG image of the current weather in Austin, TX.

Uniform Interface sub-constraints

Resource identification is one of the four interface constraints which basically requires the same authority whom maintains the reference to a resource to also be responsible for preserving meaning of that resource. The idea here is the creator of a resource will be the best agent for naming that resource. This promotes links to resources that don’t change over time even if the representations for those resources change.

The second interface constraint states that resources must be manipulated via representations. A client has no access to a resource directly, it can only send and receive representations from the server.

The self descriptive message is the third interface constraint. It states all messages must include metadata which describe the meaning of the message. This means the methods used to invoke the message must be standard and agreeable between the client, the server, and the intermediaries in between. This constraint is also related to the stateless constraint in that no state on the server is permitted when interpreting a message.

Hypermedia as the engine of application state, otherwise known as the hypermedia constraint, is the last and probably most often overlooked constraint in the REST dissertation. Simply put, the steps in a REST architecture must be invoked through hypermedia. More specifically, in order for an application’s process to move a step, it must invoke that next step from a reference inside the current representation. The next step should not be determined from somewhere outside of the currently received representation. For example, if the current representation is an html document, the next legal steps would be links in that html document.

The next article will talk more in depth about how these constraints relate to each other and how they are implemented in the HTTP protocol.

Web Requirements Part 4

2011-10-10

The requirements of the web are on a different scale than any other application to date. Even calling the web an application sounds weird, because so many people use and contribute to it. Even so, the Fielding dissertation calls the web the worlds largest distributed application. As such, it has unique requirements for its domain.

In addition to the broad requirements needed by network applications, the Fielding dissertation brings attention to four key domain-specific requirements for the web. The web must have a low barrier of entry for newcomers, be extremely extensible, support distributed hypermedia, and be internet-scale. Any architecture designed for the web, such as REST, is designed to obtain these requirements by choosing the appropriate trade-offs accordingly.

When it comes to the web it’s hard not to think of its large scale nature. When you connect something as big as the web into one unified system numerous challenges arise. When thinking of the web as a whole, the enormous load of traffic stands out as an almost insurmountable challenge to software. From the perspective of an individual software component, a design that serves 50 simultaneous connections may work for some levels of popularity but most certainly not others. A more subtle problem is the competing and sometimes political goals accompanying different organizations that want their underlying software to cooperate. The web is a large decentralized authoring platform that must also allow for the various authors with different levels of trust to fail without breaking other parts of the system. This is what the paper calls the internet-scale requirement.

Loosely related to the internet-scale requirement is the requirement for extensibility. With a large user and author base, the desired content will change over time almost assuredly. When the requirements change for some portion of the system that portion will need to change without breaking everything else.

A low barrier of entry was needed to increase adoption of the proposed HTTP standard. The paper maintains that hypermedia was chosen because of its simple nature concerning authoring and its compatibility with search services. Hypermedia could also be made to span multiple locations. Having hypermedia’s controlling information, such as when to cache, in remote locations facilitated user-perceived performance and served the internet-scale requirement as well. Given the relatively large size of distributed hypermedia messages, the web’s architecture needed to be designed for large, coarse grained messages.

Using the network-based application and web requirements as the context, my next article will finally review the REST architecture itself.

Network Applications Part 3

2011-10-08

If you have been following along in my previous articles about Roy Fielding’s REST Dissertation, you should have a very high level understanding of the paper as well as a grasp on his view of architecture. In this article I’ll go over the paper’s concept of a network application while highlighting its unique challenges.

Software is often given categorizations such as network software or application software. The paper makes a clear distinction between those two types, and then denotes a third type of software. Network software is defined as any software that moves bits over a connection without knowing what those bits are. Application software, on the other hand, takes into account the overall goals of a user. A network-based application would be software that takes into account the goals of a user and is mindful of how its bits move over the network. The REST dissertation concentrates on the network-based application domain.

There are almost always consequences in the form of trade-offs (compromises) when obtaining an architectural property. This characteristic makes architecture evaluation very elusive when coupled with a large number of desirable properties, as is the case with network-based applications. There are twelve architectural properties that are used to compare trade-offs between architectural styles in the paper. The following is a brief description of them.

Performance, particularly user-perceived performance, is revisited often throughout the paper. Some of the most prominent constraints, e.g. statelessness, in the REST architecture were done for performance reasons. Statelessness also addresses the problem of partial failure and reliability, which is a among the most significant challenges in network based software. Scalability is closely tied to performance in a network application because you want to be able to reap greater performance when adding more resources. This is especially important when considering Internet-scale applications which can have unanticipated load.

Simplicity and modifiability are needed to understand the application as a whole, and to be able to change it once it is deployed. In a system that can be deployed to more than tens of thousands of servers, stopping the system in order to evolve or extend it would be inefficient and probably futile. With a large user-base, it’s more practical to let some users customize a service on the client side. Still others may need to configure a server side component to point to a new sub-component. Optimally, it is desirable for components to be reusable without modification, in the same environment, for new purposes. If the code can be used in a different environment, it is said to be portable.

Visibility allows the application to play nice with the network. With high visibility, components can be made to assist with caching and therefore performance. This is because the messages sent between components are easily accessible (hence visible) to intermediaries on the network. Firewalls are network components that benefit from visibility in an application’s architecture by increasing security through inspection of messages.

Together these properties form the basis for comparing network-based application architecture styles. In the future I’ll revisit these properties individually while reviewing some of the more notable styles in the paper. In my next article we will take a look at the requirements for the web.

Architecture Methodology Part 2

2011-10-05

In my previous post, I gave a high level overview of Roy Fielding’s dissertation on Architectural Styles and the Design of Network-based Software Architectures (more commonly known as the REST thesis). In the paper a heady definition of architecture is put forward in which it must, among other things, be a run-time description. An architecture is not the organization of source code in text files. It also is not what you wish your run-time to be. It is a description of an actual application at run-time. This is in contrast to a style which is a group of constraints that induce properties. Styles, constraints, and properties are used to give a novel description of components. Together these concepts are used to abstract away unneeded details while focusing on desired affects in a system. I’ll try to lift some simplified definitions of these concepts from the paper and put them here.

In the paper’s methodology, elements are comprised of components, connectors, and data. A component is simply a transformer of data. A connector describes what glues the components together. Data is transformed by components, and transferred through connectors. If you are familiar with COM, DCOM, CORBA, or other component descriptions notice how the concept of data therein is usually absent or only vaguely described. This has important implications if the data being transferred through the system has a significant impact on the system. This fact is revisited throughout the paper specifically when talking about self describing messages.

An architectural style is a grouping of constraints on elements (components/connectors/data) that induce properties. A property (also known as a quality or attribute) is a desirable (or tolerable level of an) ‘ility’. Some examples of an ility would be extensibility, evolvability, or performance. A constraint produces a property in the system. An example of a constraint would be adhering to the principle of separation of concerns, thereby separating the user interface from business logic, in order to increase maintainability, evolvability, and possibly security. A software engineer can take a style’s constraints along with their respective properties and compare them to a run-time to see if it matches. A style can also be compared to other styles by using constraints and properties as a ubiquitous language to discuss architecture. Comparing the design trade-offs makes explicit the hard choices between architecture styles.

When focusing on a style (at least in a network application) it is useful to concentrate on one of three views of that style. A view is a perspective of an implemented style. A processing or component view would focus on how data is transformed in a component. A data view concentrates on the forms of data while abstracting away how and where the data is transformed. A connection view highlights state as well as interactions between components.

In order to evaluate a design for any architecture you must first decide on the type of application (and therefore that application’s functional requirements). This limits the categories of architectures being considered. For instance, a text file editor usually doesn’t need to be network aware and therefore doesn’t need a network based architecture. After the application domain is determined, the non-functional requirements or properties need to be made explicit. An example would be an application service that starts with 10 requests per second but is expected to grow to eventually receive 10000 hits per second. This application would have a non-functional requirement of scalability.

Each one of these concepts deserves its own article so I’ll revisit them later. In my next post I’ll go over the paper’s view on network applications and the implications of networks on software architecture.