Late Bloomer

Almost 15 years before Tim Berners-Lee and Robert Cailliau started up the first World Wide Web (WWW) server, James E. White published the idea of the Remote Procedure Call (RPC) with Request for Comments (RFC) 707 [1] in 1976. The idea was for an RPC to allow a computer to run a function on a remote computer. Machine-to-machine communication (M2M) thus became real.

In contrast, while developing the HTTP protocol and the hypertext markup language (HTML), Berners-Lee was thinking of human users who navigate globally distributed documents – that is, the man-machine interface.

The main difference between the two architectures is how the client approaches the resources on the server. RPC requires the client to possess very specific knowledge of how to access remote resources on the server. Here, the developer needs to study the protocol specification in detail to understand the meaning of the data elements for input and output.

On the web, however, a visitor is unlikely to read the instructions for a web page before using the service, thanks to the text-based HTML format. Users intuitively conclude how to use a web page from what they see on the page itself. Machines do things differently. In M2M communication, the developer needs to define how the machine interprets the data in the form of specifications.

Back in 2000, Roy T. Fielding, in his highly readable thesis [2], called for a drive to establish the four principles of high scalability, ease of usability, extensibility, and self-descriptive hypermedia as the basis for an efficient web architecture and to implement it consistently in the form of Representational State Transfer (REST).

Machine Talk

The inventors of the WWW quickly realized that humans were not the only ones who could take advantage of the hypermedia world. To avoid the problems inherent in the arbitrary approach of hypermedia, the developers imposed certain restrictions; for example, HTTP was designed to be a stateless protocol, which means the server does not need to worry about which point its clients have currently reached in the workflow. In this way, a server can manage many clients with little effort. At the same time, the protocol is kept very simple; it includes only a few request methods (verbs), such as GET, HEAD, and, depending on the server, the optional POST, PUT, DELETE, OPTIONS, TRACE, and CONNECT [3].

The key to the success of the web was the hypermedia format, which combines application logic and presentation. With the help of HTML interfaces, servers deliver the clients for their own web applications. The HTML format also makes it possible to combine resources so that users can navigate the application interactively and jump from HTML page to HTML page.

These relatively clearly defined constraints and the intuitive logic of the web also should work really well for the machine interface, but in practice, only a few APIs take this into account. Although increasing numbers are using the HTTP protocol, you will usually find a mix of protocol logic and application logic.

Developers also typically need to spend time learning the data format, which is far removed from the self-explanatory HTML navigation through clearly marked resource linking.

REST to the Rescue

Although it has been around for nearly two decades, REST has only seen use in the industry for a few years. One reason is because of the way developers traditionally design APIs. Often, architects define the perspective at too early a stage: The focus could be on the front end, or the database design, or the data format. As a result, the development work produces hard-to-understand and mutually incompatible APIs that sometimes offer very similar functions. REST can help resolve these redundancies and allow a sustainable operating concept.

Fielding sees the key to the REST approach as an additional service layer between components, with a view to making the user perspective independent of the specific technical infrastructure to the extent possible. Abstracting the technology, while taking into account the user's point of view, inevitably simplifies system architecture from the client's perspective. As a result, an external developer can understand more easily how a client communicates with the application.

All told, Fielding thus wants to simplify the system architecture to make the operation and development – not only of the application, but also of thousands of independent clients – efficient and sustainable. Abstraction no longer occurs exclusively at the database (Postgres, MySQL) level and web server (Apache, Nginx) level, but also in the data formats, which are based on standards, conventions, RFCs, and registered media types.

Fielding defines four necessary conditions for REST, as follows:

• It must uniquely identify relevant resources.
• It is only allowed to change resources through abstract representations.
• It provides self-explanatory descriptions.
• It relies on the hypermedia format, which controls the application state.

Probably because Fielding's description was quite abstract, practical implementation only arrived after a long wait. Only now, after many API developers have gone through the painful experiences that Fielding anticipated, are developers translating his ideas into practice.

Finally Tangible

Today REST – in the context of the web – can be defined as resources, including database entries, images, documents, or other electronic data, that can be addressed via unique Unified Resource Identifiers (URIs) that do not need to be user-readable, and definitely not in a machine-to-machine interface. However, it should be possible to clearly match a URI to a resource.

The developer does not manipulate resources such as database entries directly, because REST first abstracts them in a presentation layer, thus decoupling the data from the database design.

The data formats used here contain both all the information and functions to cover the application logic. Because REST relies on standards, the application behavior follows well-known mechanisms, and the interface more or less describes itself. For example, the method attribute is used in a form; the use of href to link to another resource simply makes sense. Last but not least, HTML, with its well-defined elements, including links and forms, guides the web application from one controlled state to the next.

REST API Design

REST is merely an architectural model that does not specify the concrete implementation. To begin, you implement the following four steps:

1. Translate the application logic into the REST schema. This step requires a certain effort, but it saves half the battle. The extra effort pays off when you need to introduce external developers or new employees to the API logic, which also is often a key acceptance factor, especially for open APIs with a widespread developer community. A structured design process helps with the remaining steps [4] [5].
2. Design the interface model. The model determines what data is handled by the interface and what state the application assumes after each step. The relations between the roles involved in the process emerge here and appear as links later.
3. Define standards for the identifiers (e.g., links or media types) to the extent possible. Countless registered identifiers are already available [6]-[8]. An API will integrate in a far better way if it behaves as expected or in a known way. To match the identifiers closely, you then select the media types, often with the need to iterate through steps 2 and 3 multiple times until the results work out.
4. Document your own interpretation. As a result, you then ideally have an application profile that consists of a registered standard profile with minor adjustments. Only in this step do you get around to implementing the API.

OData

REST works especially well in the case of Internet applications with large numbers of users and million-fold parallel access, which explains why the major tech providers, especially, are increasingly relying on REST. In the field of data formats, the OASIS Open Data Protocol (OData) Technical Committee [9] endeavors to achieve standardization. Many major vendors support the initiative, including Red Hat, Microsoft, IBM, and SAP.

OData is a relatively comprehensive protocol and requires some training. The REST model is based on HTTP, the Atom Publishing Protocol [10], and JSON [11]; it consistently uses URIs to address resources. The deliberately varied use cases include relational databases, filesystems, and content management systems, as well as traditional websites.

In February 2017, the ISO Joint Technical Committee accepted the OASIS OData Standard for Open Data Exchange (ISO/IEC 20802-1:2016 [12] and 20801-2:2016 [13]). Against this background, OData is more of an industrial policy program and interesting in the context of large corporations or authorities.

OpenAPI

The Open API Initiative [14] focuses on vendor-neutral API format descriptions [15] on the basis of the Swagger specification [16]. The description language is designed to enable the development of APIs that both humans and machines can understand, independent of the programming language. The specification is available under the Apache 2.0 license; its supporters include Google, PayPal, Red Hat, and Microsoft. The many supporting tools [17], such as the YAML Swagger Editor shown in Figure 1, prove particularly useful.

The Swagger Petstore (Figure 2) [18] is another highlight; a sample server demonstrates the specification and allows experiments. The OpenAPI specification is documented very clearly, which significantly shortens the learning curve and makes it suitable even for small projects.

RAML

The RESTful API Modeling Language (RAML) [19] takes a very similar approach to OpenAPI. The initiative is backed by MuleSoft, VMware, and Cisco, among others. RAML promises to support the entire life cycle of an API – from design to rollout – and succeeds in doing so with the help of its toolset and well-prepared documentation [20].

Much like the YAML Swagger Editor, MuleSoft has developed a RAML web API Designer (Figure 3) [21]. The designed API elements can be tried out directly through a mocking service. MuleSoft loads the API design in a web service and executes it. Alternatively, MuleSoft offers an open source development environment named API Workbench [22], although it is under a proprietary license.

RESTful with Ruby and Sinatra

As a simple example, I'll look at a RESTful API for managing a list of names. The RAML API Designer helped with the design. To begin, I designed the API and tested it in the RAML API Designer in a mocking environment to see whether it formally provides the right inputs and outputs. Next, I wrote the first line of code on the basis of RAML specification 0.8 (Listing 1). Thanks to the clear-cut YAML-based presentation, the developer can focus fully on API functionality, which keeps the peculiarities of a language out of the process. A tutorial [23] offers a good introduction to best practices for developing a REST API.

01 #%Raml 0.8
02 title: Contacts
03 version: 0.1
04 #baseUri: http://www.rve.com/contactshelf
05 baseUri: https://mocksvc.mulesoft.com/mocks/d4c4356f-0508-4277-9060-00ab6dd36e9b/contactshelf
06 /contacts:
07   get:
08     description: Retrieve list of existing contacts
09     responses:
10       200:
11         body:
12           application/json:
13                 example: |
14                   {
15                     "data": [
16                       {
17                         "id": "1",
18                         "name": "Rheik",
19                         "link": "http://localhost:4567/contacts/Rheik"
20                       },
21                       {
22                         "id": "2",
23                         "name": "Paul",
24                         "link": "http://localhost:4567/contacts/Paul"
25                       },
26                       {
27                         "id": "3",
28                         "name": "Emil",
29                         "link": "http://localhost:4567/contacts/Emil"
30                       }
31                     ],
32                     "success": true,
33                     "status": 200
34                   }
35   /{name}:
36     get:
37       description: Retrieve a specific contact
38       responses:
39        200:
40          body:
41             application/json:
42               example: |
43                 {
44                   "data": {
45                     "id": "1",
46                     "name": "Rheik",
47                     "link": "http://localhost:4567/contacts/Rheik"
48                   },
49                   "success": true,
50                   "status": 200
51                 }
52   /new:
53     post:
54       description: Add a new contact
55       body:
56         application/json:
57             example: |
58               {
59                 "data": {
60                    "name": "Franz",
61                    "link":  "http://localhost:4567/contacts/Franz"
62                  }
63               }
64         responses:
65           200:
66             body:
67               application/json:
68                 example: |
69                    { "message": "Contact transferred correctly." }
70   /{id}:
71     delete:
72       description: Delete a specific contact
73       responses:
74         200:
75           body:
76             application/json:
77               example: |
78                 { "message": "Contact id = <<resourcePathName>> deleted." }
79         404:
80           body:
81             application/json:
82               example: |
83                 { "message": "Contact id = <<resourcePathName>> not found." }

The data for the service is stored in a SQLite database. The script in Listing 2 fills the database with three sample entries from the command line and then lists the entries.

$ sqlite3 contacts.sqlite "CREATE TABLE contacts (id INTEGER PRIMARY KEY, name, link);"
$ sqlite3 contacts.sqlite "INSERT INTO contacts (name, link) VALUES ('Rheik', 'http://localhost:4567/contacts/Rheik');"
$ sqlite3 contacts.sqlite "INSERT INTO contacts (name, link) VALUES ('Paul', 'http://localhost:4567/contacts/Paul');"
$ sqlite3 contacts.sqlite "INSERT INTO contacts (name, link) VALUES ('Emil', 'http://localhost:4567/contacts/Emil');"
$ sqlite3 contacts.sqlite "SELECT * FROM contacts;"
1|Rheik|http://localhost:4567/contacts/Rheik
2|Paul|http://localhost:4567/contacts/Paul
3|Emil|http://localhost:4567/contacts/Emil
$

The Ruby script in Listing 3 shows the program for a Sinatra server that provides a simple REST API, which is only RESTful in its approach because the JSON format used in the example is not standardized, referenced by a schema, or genuinely based on hypertext. The corresponding hypermedia schema would be used here, but it was left out of the example for the sake of clarity.

01 require 'rubygems'
02 require 'sinatra'
03 require 'sinatra/json'
04 require 'uri'
05 require 'active_record'
06
07 ActiveRecord::Base.establish_connection(
08   :adapter => 'sqlite3',
09   :database =>  'contacts.sqlite'
10 )
11
12 class Contact < ActiveRecord::Base
13 end
14
15 get '/contacts' do
16   @contacts = Contact.all
17   @response_message = {data: @contacts, success: true, status: 200}
18   json @response_message
19 end
20
21 get '/contacts/:name' do
22   @name = params['name']
23   @contact = Contact.where(name: @name).first
24   @response_message = {data: [@contact], success: true, status: 200}
25   json @response_message
26 end
27
28 post '/contacts/new' do
29   @data = JSON.parse(request.body.read)['data']
30   puts @data.inspect
31   if @data['name'] and @data['link'] then
32     @contact = Contact.new(name: @data['name'], link: @data['link'])
33     @contact.save
34   end
35   @response_message = { "message": "Contact #{@data['name']} created." }
36   json @response_message
37 end
38
39 delete '/contacts/:id' do
40   @id = params['id']
41   @contact = Contact.find_by_id(@id)
42   if @contact then
43     Contact.delete(@contact.id)
44     @response_message = { "message": "Contact id = #{@contact.id} deleted." }
45   else
46     @response_message = { "message": "Contact id = #{@contact.id} not found." }
47   end
48   json @response_message
49 end
50
51 options '/' do
52   # Currently supported HTTP verbs
53   response.headers["Allow"] = "HEAD,GET,POST,DELETE,OPTIONS"
54   204
55 end

Easy Access

Assuming you are in the contacts.sqlite database directory, the Sinatra server can be launched by typing ruby contacts.rb. The client role is played by the versatile curl on the command line. To list the existing entries in the database, curl accesses http://localhost:4567/contacts, which then returns the data in JSON format.

The -v option displays all the HTTP parameters, from which you can specifically tell that the content type is application/json (Listing  4). Curl picks this up with curl "http://localhost:4567/contacts/Rheik", which corresponds to the get '/contacts/:name' method in the Sinatra script and /{name}: in the RAML draft.

01 $ curl -v "http://localhost:4567/contacts"
02 *   Trying ::1...
03 * TCP_NODELAY set
04 * Connected to localhost (::1) port 4567 (#0)
05 > GET /contacts HTTP/1.1
06 > Host: localhost:4567
07 > User-Agent: curl/7.51.0
08 > Accept: */*
09 >
10 < HTTP/1.1 200 OK
11 < Content-Type: application/json
12 < Content-Length: 382
13 < X-Content-Type-Options: nosniff
14 < Server: WEBrick/1.3.1 (Ruby/2.3.3/2016-11-21)
15 < Date: Thu, 30 Mar 2017 18:48:49 GMT
16 < Connection: Keep-Alive
17 <
18 * Curl_http_done: called premature == 0
19 * Connection #0 to host localhost left intact
20 {"data":[{"id":1,"name":"Rheik","link":"http://localhost:4567/contacts/Rheik"},{"id":2,"name":"Paul","link":"http://localhost:4567/contacts/Paul"},{"id":3,"name":"Emil","link":"http://localhost:4567/contacts/Emil"}],"success":true,"status":200}
21 $

Curl creates a new entry with a call to the POST method (Listing 5), but the program here breaks with REST logic. A more intuitive approach would be a POST to the URL of the contact resource http://localhost:4567/contacts/Jim; however, many developers expect a fixed path (/new), if the resource does not yet exist.

$ curl -H "Content-Type: application/json" -X POST -d '{"data": {"name": "Jim", "link": "http:://localhost:4567/contacts/Jim"}}' http://localhost:4567/contacts/new
{"message":"Successfully created contact Jim."}

It would be formally correct to send the POST to /contacts. The best idea is for developers to find a solution that catches the eye.

The call

curl -H "Content-Type: application/json" -X DELETE      http://localhost:4567/contacts/6

deletes the entry with ID 6. In contrast to the name, the ID is unique.

With the options method (Listing 3, lines 51-55), the server tells the user, on receiving an appropriate request, which HTTP verbs it supports ("Allow"). The answer requires the curl -i parameter, because the output will not otherwise appear (Listing 6).

01 $ curl -i -X OPTIONS http://localhost:4567/
02 HTTP/1.1 204 No Content
03 Allow: HEAD,GET,POST,DELETE,OPTIONS
04 X-Content-Type-Options: nosniff
05 Server: WEBrick/1.3.1 (Ruby/2.3.3/2016-11-21)
06 Date: Thu, 30 Mar 2017 7:11:56 PM GMT
07 Connection: Keep-Alive
08 $

Conclusion

REST is an abstract idea designed to better structure the behavior of a software architecture. The approach demands discipline and requires an additional level of abstraction on top of the desired solution. Not everyone is enamored, and how meaningful this overhead is depends on the API's intended use. How long do the developers intend to maintain it? How quickly do you want to be able to introduce it to new developers? How compatible should it be with other solutions? These are just a few of the fundamental issues.

The sample API on the Sinatra server is designed to illustrate the abstraction. To create a full RESTful API, developers would need to elaborate the RAML design and make a sensible choice for the data format standard.

The advantage of REST is the concept of the "Hypertext As The Engine Of Application State" (HATEOAS). Roughly this means that, based on the existing hypertext, any client should know the current application state and be able to enter the next state via the hypertext mechanisms.

The sample does not implement this concept in a satisfactory way. The client can tell from the response how to access a single contact, but it does not learn how to create it or how to structure the data to do so. The next step would be to integrate this information into the format itself.

Some standard formats offer to add POST templates or schema descriptions to the protocol. It makes a lot of sense to look at the registered formats and protocols such as IANA [6], Schema.org [7], and Alps.io [8].