February 13, 2011
Like many Ruby on Rails developers, I have been wrestling with the concepts of REST. While the basic structure of a RESTful interface as implemented in Rails is fairly easy to grasp, I have discovered significant subtleties and difficulties to RESTfully implementing a real-world RoR web application that introductory books with their sample applications do not adequately describe.
My fundamental realization is that there is a significant difference between a web application server that serves HTML pages for a human client (mediated by a web browser) and a web service for other machine clients. REST is fundamentally an API schema for web services which has been extended for web application development in Rails. It works well for applications that are fairly simple views onto an underlying database, but the statelessness and limited verb set of true REST are impossible to maintain as the workflow and data interactions of a web application become more complex.
It is simply impossible to implement a complex Ruby on Rails web application that is completely RESTful. Even “canonical” RESTful implementations deviate from REST in some important ways. (There is nothing very RESTful about storing per-user information in cookies or sessions, for example.) Every Rails developer needs to make his/her own compromise between:
First, a brief recapitulation of REST:
REST stands for “Representational State Transfer”. Let’s deal with each of these terms in turn:
Richardson & Ruby in their book “RESTful Web Services” (see below), among others, describe four major qualities of REST:
Note: POST is neither safe nor idempotent, but it behaves predictably when used RESTfully.
Maintaining RESTfulness in a Ruby on Rails application is not difficult in a fairly simple resource-oriented application, which consists primarily of presenting views of individual resources and collections. (A “resource” need not map directly to a single model, but it usually does in such straight-forward applications.) The RESTful schema can become a lot more difficult to apply when the application has to maintain a complex state to be able to present a “wizard-like” view progression or deal with complex resources.
Applications with more complex forms of state generally must store it either in cookies or a session. And that’s not RESTful. Similarly, a web application may have a resource which can not be manipulated using the restricted HTTP four-verb set. Extra non-standard operations can be introduced, which is non-RESTful, or auxiliary resources can be introduced to express the operation. Not only does this substantially complicate the web application, but the resulting multi-stage resource manipulations must often be serially performed by the user (for thin clients, at least.)
It is worth noting that neither of these issues apply to web services communicating with machine clients, only to web applications communicating with humans through thin client browsers.
The ability to manage application state is important to any web application that is seen as a coherent collection of inter-related web pages. But a thin client browser can hold very little state (essentially only the currently displayed URL), and a RESTful web application is forbidden from maintaining such application state. This can make fully RESTful development of human-facing web applications difficult or impossible (depending upon how much of a purist you are.) If your application is truly RESTful, it embeds no application state, meaning that the response to an HTTP request is totally dependent upon the request and the current state of the target resource. Maintaining some kind of history of previous requests and altering the response to the current request and the redirection to the next request based upon it is an impermissible use of application state. A RESTful web server doesn’t do that.
And unless we are using something like Ajax to build an intelligent client-programming layer into a web browser, the only application state that the client knows about is completely represented by the current URL. Even with the assistance of user-selectable links embedded in a page and workflow sequencing provided by server-generated redirects (see below), this stark restriction on application state greatly limits the richness of a RESTful web application.
Cookies and sessions are mechanisms by which per-user data can be maintained, either on the client browser, in a session on the application server, or a combination of both. While they appear in many RoR and other web applications, their usage violates RESTfulness, a fact which tends to be glossed over.
The non-RESTful nature of cookies and sessions stems from the fact that they are fundamentally used to maintain per-user application state accessible to the server, which is exactly what a RESTful server is not supposed to do. A RESTful server is supposed respond to a request based only upon its URL and the current state of the target resource. The basic purpose of cookies/sessions is to allow the server to respond quite differently based upon these hidden values.
So the use of cookies and sessions is definitely non-RESTful. That said, is there any mechanism other than a thick Ajax/JavaScript client layer that can hold application state? The answer is “no.” For that reason, any web application that requires more than the workflow assists than links embedded in pages and server redirects must use this feature, even if it otherwise attempts to be RESTful.
Is there anything that can be done to minimize the non-RESTfulness of cookies/sessions? Not really, in my opinion. I would suggest using cookies only as needed to implement a user session, and using the session only to implement a UserSession singleton resource, complete with controller, model, and possibly views. That way a UserSession resource can be accessed both internally and by the client similar to an ordinary resource. It’s just that the backing data is at least partially stored in the client. And if UserSession were to be accessed in isolation, it would truly be a RESTful resource, regardless of its storage location. But it’s not. Its purpose is to be secretly accessed by the server while processing requests on genuine resources. And that’s what makes it non-RESTful.
A web application that employs user sessions or cookies may employ RESTful principles, but it certainly can not be said to be fully REST-compliant. Still, there really is little else is out there to assist a web application in maintaining application state.
I have read the mistaken notion that a session somehow becomes RESTful if stored in a client browser using a cookie. Not so, for the simple reason that the browser has no way to control this state except to delete it entirely. The browser is merely providing an auxiliary mechanism for storing server state, because only the server can read or manipulate it. Thus a user session is non-RESTful, no matter how stored.
The second major problem that a web application which must manipulate a complex resource is how to do so using only the four permitted HTTP verbs GET, POST, PUT, and DELETE. This is a pretty restricted set, yet REST demands that developers implement all manipulations on resources, no matter how complex, using just those.
The recommended technique is to convert a proposed action into an additional resource. For example, suppose we need to implement a “close” operation on a bank account which is different than deleting the resource. Instead of taking the easy way out and defining a non-REST “close” action for the account resource, we convert the action into a resource and define a new account_closure resource.
To close an account, we would create a new account_closure resource as belonging to the account to be closed. Then we might update the account_closure to actually effect the close action. In addition to updating such account_closure attributes as closed_at, update would also likely change a status attribute in the now closed account. Very likely this account_closure resource should be maintained for audit trail purposes, but if it were not, one could then perform the delete action on account_closure.
Instead of a simple non-RESTful added “close” action, we now have a more complex RESTful set of resources.
Whether an action such as “close” is implemented as an extra action or as an account_closure resource also has an impact on web application design. If implemented as an action, the client need only send a PUT to a URL specifying the account resource and the non-standard action (E.g. “accounts/23/close”).
But if “close” is implemented as an account_closure resource, up to three separate requests are necessary to a) create the resource, b) perform the action, and c) to clean up after it (i.e. delete the resource.) A specially-programmed software client (or an Ajax or JavaScript-augmented web browser) will have no trouble performing such a sequence. But what about an ordinary web browser that just renders the HTML served by the web application?
Here’s how a RESTful account “close” action might be implemented by a thin client browser:
Not too bad, but it requires two-steps of user interaction, when only one might be better for some types of interactions. Additionally, if the auxiliary resource is to be deleted afterward, this is best done by an independent daemon process that periodically deletes stale records from the database. This is still more work.
My own feeling is that an auxiliary resource is required when a transaction on two or more resources must be performed, i.e. debiting one account and crediting another. (Remember that a RESTful request can specify only a single resource as the target for an action.) However, when this isn’t necessary, other techniques that only modestly violate RESTful principles might be better. See “The :action_type Parameter — a Substitute for Special Actions” below for one such technique.
Note that implementing actions as auxiliary resources is more flexibly done using an Ajax layer on the client side to transparently handle the necessary extra chit-chat.
The workflow of a web application involves a constantly changing application state as the user interacts with the server. Remarkably, a thin client web browser without an Ajax/JavaScript assist has almost no capability to alter application state on its own. Rather, it relies on several features implemented by the server to accomplish this. The server can insert links into web pages to allow the user to control the workflow, and it can redirect the browser to display a particular page after form submission. But that is about it. These devices work so successfully for most web applications that developers tend to not pay much attention to the limitations.
These features are generally orthogonal to the REST concept, but they are essential for a human-consumed web application. Without them, a user would be unable to create or update resources or navigate the application except by laboriously entering URLs manually. Their existence compensates for the lack of application state in a RESTful web application server, at least for simpler applications.
It is worth noting that only a web application requires such mechanisms. A web service where the client application fully controls the application state has no need of them.
The current state of a RESTful application is completely defined by the URL currently displayed by the browser. (Client-side technologies such as JavaScript & Ajax complicate this picture.) The way a human can change the application state (other than to fill out a POST-able form and press “submit”) is to enter a different URL. Links may be embedded into pages sent in response to a GET request to allow the user to automatically submit a different URL. Links have been called “levers of application state”, meaning that they are helpful suggestions provided by the web application of possible URLs that the human might wish to navigate to.
A redirection is generally provided in the response to a PUT or POST request to indicate the next action that the application should issue. A response to each HTTP request is sent back to the originating client and incorporates a response code. A 300 Redirection series has a fundamentally different function in a web service versus in an interactive web application. A redirection code sent by a web service may tell the machine client where to obtain a response to a successful request or where to instead direct a request. A redirection code serves instead to implement a degree of application workflow when sent by a web application server.
For example, consider a POST or PUT request dispatched by a web browser to send data to the server. Were the server to send a “200 – OK” for a successfully completed request, the browser would display this code, but it wouldn’t have anything else to present. And the human user would be stuck. Her only next action would be to manually enter a URL which represents a next stage of the application, something decidedly awkward. A web application instead helpfully sends a Redirection status with a target URL indicating the next page to display. This is often a redirect to the index (for the collection) or show (for the resource) following create or update of a resource, but the target could be anywhere.
As I have telegraphed above, there is one way out of the dilemma of developing an effective web application while fully adhering to RESTful principles: to abandon any kind of application state server support, and put it all in an Ajax/JavaScript layer implemented in a thick client browser. That way the server layer becomes a completely RESTful web service, capable of also serving other kinds of clients.
An Ajax/JavaScript client layer can store state and act as an intermediary between the browser UI and the web server. Not only can this be a fully RESTful solution, if the client layer programming is thick enough, the web server can drop most aspects of its former life as a web application and simply become a web service API. The client could render a web page from XML output by the web server, or it could still rely on the server to deliver complete or partial HTML pages. Either way, the client stores the application state and controls the workflow sequencing. The only kind of web application support that the server need retain is to deliver the client layer JavaScript code to the browser.
It is not necessary to implement such an intelligent client layer entirely in the browser, however. It might be useful to implement resources on the server that maintain application state. E.g. a Dashboard model and associated controller and views could be implemented to present the heterogeneous content of a typical dashboard screen. Such server-implemented resources can reduce the amount of code required for the actual client layer. If only the client interacts with such resources, and not the server behind the scenes, we can still make the argument that this is RESTful, even though these resources actually deal with application and not problem domain resource state.
POST permits arbitrary data to be sent (“posted”) to the server. A few uses of POST are RESTful; many are not. A RESTful use of POST specifies a resource under which a new “subordinate” resource is to be created plus attribute values for the new resource. The classic POST usage example is to create a resource (e.g. a Recipe) underneath a collection of similar resources (e.g. a set of Recipes.)
A non-RESTful usage of POST is what Richardson & Ruby (see below) term the “overloaded POST”. Here the target can be anything, and the posted field values can represent anything, operations, modifiers, and field values for one or more resources. This is essentially an RPC type of interface.
Note: POST and PUT are very similar. PUT changes the state of a (usually) existing resource specified by the URL, while POST creates a new resource subordinate to the specified resource and returns its identifier. (PUT could also create a new resource if the web service allows the client to specify the resource id.)
Another well-promoted aspect of RESTful Rails is the ability to serve multiple representations of a resource in a single application by weaving code to serve different representations into controllers using “respond_to”. In this manner, a single application could serve both human-readable (html) and machine-readable (xml) representations, for example.
However, a service and an application have rather different features and characteristics, and implementing both in the same line of RoR code is in my opinion wise for only fairly straightforward implementations. The requirements for each representation may be substantially different in more complex applications, or they might diverge in the future. This concept works well in sample applications; it might not work at all well in complex real-life situations.
For example, still giddy from the RESTful Kool-Aid I had drunk, I was ready to implement the mobile representation of a consumer web site in this fashion. However, another developer on the project warned me against it, and this proved wise. The main line of development had a lot of complex interwoven controller code for features that the mobile version would not use, and injecting mobile representations into these controllers would buy little to compensate for the inevitable mutual impediments to both lines of development. I instead implemented the mobile representation with its own controllers, using many of the models, helpers, and other infrastructure of the main line of code. In this way I narrowly avoided the disaster that otherwise would likely have ensued from the tight coupling.
The relationship of New and Edit to the RESTful actions can be confusing. The fact is that, while not anti-REST, New and Edit are custom actions that have been added to the Rails REST implementation to compensate for REST’s web services bias when developing a web application.
New and Edit are both based upon the GET request. While a standard GET on a collection resource returns a representation of the collection, GET on a collection modified by appending ‘/new’ instead returns an empty form, which can be POSTed to create a new resource. Similarly a standard GET on an individual resource returns a representation of it, but GET with ‘/edit’ appended instead returns a form filled in with the current values. This form can be PUT to update the resource.
Both types of forms essentially tell the human user how to create or modify a resource, something dispensable in an web service for machine clients, but vital when implementing a web application for human consumption.
The point is not that New and Edit are bad, but that they are improvisations to compensate for a lack of web application support in the REST schema. And if RESTful Rails requires such compensations, maybe your web application could benefit from similarly benign improvisations.
In a recent project I had to update a model in a number of different ways. More critically, I had to redirect to different views depending upon which workflow sequence had invoked the update action. While I considered a number of different solutions, the one I hit upon was to include an :action_type parameter with each non-standard PUT request to the resource. Not only was I able to perform slightly different update processing for each :action_type, I also used the parameter to specify the post-update redirection. While this seems to be a form of what Ruby & Richardson call the “overloaded POST”, in my opinion it is a benign form of it which deviates only modestly from canonical Rails RESTful practices and preserves the form of the REST “uniform interface.” While including an :action_type parameter with PUT requests may be functionally similar to defining special non-RESTful actions, I see it as a superior technique. A similar concept of providing a :view_type parameter with GET requests can be used to customize the returned HTML page.
In a naive effort to be RESTful, for one RoR project I considered maintaining application state as query parameters or hidden fields which were to passed back and forth between the server and browser and acted upon by the server when appropriate. This is actually a bad idea, for the following reasons:
Just storing application state values in a cookie or session, an honestly non-RESTful practice, is far better than trying to pass state using query parameters.
The case I have been trying to make is that it is impossible for a web application to be completely RESTful, except for fairly simple applications or those storing application state in a thick client layer implemented in Ajax/JavaScript. My two takeaway points are these:
I welcome comments and criticism of this short article. Please email me at edwin@edwinmeyer.com
The following books contain various degrees of information on REST in Rails and are listed more in less of order of recommendation. (All presented links are to the Amazon web page for the books.)
RESTful Web Services (O’Reilly, 2007) by Leonard Richardson and Sam Ruby.
I consider this an essential resource for any Rails developer who wants more than a superficial understanding of REST. Most of the concepts presented here are described in far more detail in this book. The book’s examples are almost entirely Rails-based.
Enterprise Rails (O’Reilly, 2008) by Dan Chak.
A rather iconoclastic presentation of various topics important to implementing Rails in the enterprise, this book is a good antidote to any Rails Kool-Aid that you may have ingested. Contains two solid chapters on REST.
The Art of Rails (Wrox, 2008) by Edward Benson.
I have also found this book to contain a clear exposition of various REST topics, as well as other aspects of Rails.
Practical REST on Rails 2 Projects (Apress, 2008) by Ben Scofield.
Provides a concise introduction to REST on Rails, followed by a “MovieList” RESTful service implementation. Subsequent chapters present various types of clients built to use this service, including for Facebook and the iPhone (which may be outdated by now.) I’m still reading it, but it seems useful for its specific examples.
Service-Oriented Design with Ruby and Rails (Addison-Wesley, 2010) by Paul Dix.
While REST-oriented, this 300-page book is somewhat off target for web application developers because, as the author states by way of contrasting it with RESTful Web Services: “The focus of this book is on internal services rather than external ones.” Still, this book is one of the few that cover RESTful services in any detail. It is valuable for anyone developing a web service, whether it be outward-facing to the public, or inward-facing strictly for organizational clients. Disclaimer: While I have read excerpts on Amazon.com, I have not read the book.
The following two books are among the top selling Rails books, and both include substantial sections on REST. They tend to be more “how-to” than philosophically oriented.
Agile Web Development with Ruby on Rails, 3rd Edition (Pragmatic, 2009) by Sam Ruby, Dave Thomas, and David H. Hansson.
A serviceable introduction to Ruby on Rails, the first third of the book presents Rails by building a sample online bookstore web site. The remainder of the book describes the usual Rails implementation suspects. Provides a compact section on implementing REST features, including helpful tables and examples. Interestingly, the sample application’s routes are made fully RESTful only in the fourth edition released March, 2011 for Rails 3.0.
The Rails 3 Way (Addison Wesley, 2010) by Obie Fernandez, et. al.
A massive tome that includes a lot of information on most aspects of Rails. A rather comprehensive reference, but quite light on example code. Note: This is the second edition. The first edition (which is the one I have read) was released in 2007 and covers Rails version 2.
Finally, for another view on REST purism from consultant and Rails core contributor Michael Koziarski, plus insightful comments from others, check out Taking Things Too Far: REST