Dan’s Guide to RESTful services and REST APIs

This article deviates from the standard article format that I’ve been using on this site, and my predecessor blog for the last 10 years.  It takes the form of a Guide designed to be updated with more content over time.  

As this document is a living document, there will be notes (TODO:) where I have indicated a topic needs to be expanded on.  

REST Basics

REST is an open protocol for designing web based API’s and distributed applications not relying on any specific technology or vendor.  Generally used over http and https, REST services use resources in order to provide data and functionality to clients typically in JSON format.  Identifiers can be used along with resources in order to specifically identify a single item from a resource.

Imagine a list of products each with their own product id’s.  The list of products will be accessible with a URI like:


Whereas the product with the specific product id of 12 will be directly accessible via the following:


These are both “GET” requests designed to retrieve data.

Resource Identification with Multiple Keys

As you see in the example in REST basics, a resource (item/record) is accessible through the resource URI.   For example:  https://mycompanyrestapi.com/api/products/12 will retrieve the product with the id of 12.  This can be presented in different ways, and it does not have to use the identifier of the underlying database key, however, the identifier needs to be consistent across the entire api.  Most Rest API implementations tend to use the backend key (such as database identifier) unless there are security risks in doing so.

There may be cases where a resource, in this example, a product, is not uniquely identifiable via a single key.  This could be for a variety of reasons, including backend data storage architecture, or other storage or database constraints.

There are a few ways to approach this:

  1. Use a parameter filter to further filter and identify the actual product.  Example:  /api/products/keyboards?id=12 . – I generally don’t like this approach and I don’t think it’s very useable.  /api/products/{id} will no longer return a single resource, and instead it will return a list of resources that need to be filtered further with a parameter to get the product of keyboard category with the identifier of 12.
  2. A better approach to above, and the more rest compliant approach, would be to create a routing or a url path which includes the product category in the URI.  So, your URI would be formed as follows:  /api/productcategories/{categoryid}/products/{productid} .  Essentially, rest uri’s can be designed in anyway that makes sense to you as long as the hierarchy makes sense.
  3. If product id truly uses a 2 part identifier, but you can’t find a use case to use the approach above, you can consider concatenating your 2 part key in order to create a unique key, or find some other combination of values that guarantee that the id is unique for the resource.  This means your identifier used by rest clients will not necessarily match the true identifier in the backend storage of your systems, so you will need to make sure that the same unique identifier is used across your RESTful services which use or refer to the resource in question.  With the product example, you would then have a Uri with an id such as:  /api/products/keyboard-12

I’ve run into this situation quite a few times.  We had a medium sized Azure implementation in which I came on board and lead the architecture for.  Because the design decision based on costing was to use Azure tables for some of our storage needs, we designed a list of resources with the standard Partition Key and Row Key in Azure, where the two properties uniquely identify an item.  When creating a REST Api around our Azure data infrastructure, I felt it was appropriate to use a combined key of the Partition Key and Row Key to identify the resource in the REST Api (number 3 above).

In another case, the data entities and resources were much bigger and much more complex with many more client use cases.  It was an enterprise implementation, and therefore due to requirements, ease of use, and ease of maintenance, I designed the Rest API using nested resources (number 2 above).  The great thing about this approach is that the nesting can take multiple forms.  You can have the product resource nested directly under category, and also nested directly under another resource, distributer, for example.  So you end up with multiple nesting options for clients to access the data they need.  For example:  /api/categories/{categoryId}/products/{productId} will return of specific product.  /api/distributer/{distributerId}/products/ will list all products for a given distributer and /api/distributer/{distributerId}/category/{categoryId}/products/ will list a all products for a given distributer in a given category.

Relations to Other Resources

Although there is no JSON standard for linking one resource to another, there are some generally approaches used in REST APIs.  When dealing with REST data, relationships will exist between one resource and another, and this relationship is generally signified using a URI as we see in the sample JSON below for product id 12.  There are different approaches, but quite often a links array will be included as part of the JSON data which will identify the link and provide the URI to that specific links related resources.

"productID":12"name":"A really cool stereo", 
"links": [
         {"rel":"distributors","href":"https://mycompanyrestapi.com/api/distributors/350", "action":"GET" }, 
         {"rel":"vendors","href":"https://mycompanyrestapi.com/vendors/178", "action":"GET" } 

A lot is assumed with this JSON data example.  For example, it assumes that there is only one distributor, but in the real world there could be multiple distributors and therefor the JSON could be returned in a variety of different ways depending on how the creator of the REST service designed it.  The JSON data for a product could have the basic entities for each distributor, indicating the name of the distributor along with the Id and link to the distributors URI for that distributor such as in the following example.

"productID":12"name":"A really cool stereo", 
"distributors": [{"id":350, "name":"SuperLight Distributors", "href": "https://mycompanyrestapi.com/api/distributors/350"},
                 {"id":357, "name":"Heavytown Distributors", "href": "https://mycompanyrestapi.com/api/distributors/357"},

Notice that there are some differences here in the generally approach taken to linking to other resources.  There is no standard approach to linking with JSON data, so each RESTful service will generally follow established patterns and ensure they are documented in the REST Api documentation.  Even in these two examples, there are differences in how the Uri’s are provided, both structurally within the JSON and the amount of data included.  The second example, the client will assume that it is a GET request whereas in the first example, “GET” is explicitly indicated.

Stateless Architecture

There are no sessions or session states in a RESTful service or architecture.  The client is solely responsible for managing state if necessary, and is solely responsible for providing all the needed data to the REST service in order for the request to be successfully fulfilled.

Securing Your REST Services

REST services should not store any authenticated data from the client, and although some REST Api’s do maintain authentication state for a client such as through a cookie or header token, it is a bad practice.  Generally, every REST request should contain the authentication data the service needs to validate authentication prior to executing.

Most REST services will need to be secured.  The first step in this process is authentication and authorization.

Authenticating and Authorizing a REST request

When a connection to a REST service is made, the credentials supplied to it are verified to confirm that the client making the call is verified to be who they claim to be.  Now that the client has been authenticated, they need to be authorized.  Authorization just determines if they have specific access or authentication to the resource they are trying to access.  As a best practice, securing the communication between clients (via SSL/TLS) and rest services is something that should be done, it’s even more important and mandatory when using Basic Authentication, OAuth, or another authentication scheme where the authentication credentials can be accessed in plain text.

Types of Authentication

Authentication for REST services can be handled in a variety of ways.  Here is a list of some of the more common approached to authentication.

HTTP Basic Authentication: This authentication mechanism is simple, and is part of the HTTP standard.  The implementation is just to provide a username and password in plain text as part of the HTTP header of the request.  The name of the header is “Authorization”, and the user name and password are provided in base64 encoding with a (:) separating the username and password (username:password).  Obviously, this is only secure if the communication is encrypted (SSL).  The response status from the server will either be 200 – OK, or 401 – Unauthorized.  With HTTP Basic Authentication. the username and password are sent from the client to the server on every request, and the server authenticates the client on every request.

Hash Based Method Authentication Code (HMAC):

In an HMAC authentication scenario, the security lies around a secret key that is known by both the server and client.  This key can be used to verify both the data integrity of the message as well as to authenticate.  The HMAC is calculated with a cryptographic hash function such as MD5 or SHA-1.

The components involved for HMAC authentication are the cryptographic hash function, the secret key, and the message to be authenticated, and an internal key which is derived from the original key.

To generate an HMAC, there are two hash computations involved, an internal and external.  The internal hash is produced first, and then the external hash is produced deriving from the internal hash combined with the inner key.

With HMAC, integrity of the message is kept in check, by including the timestamp, a random number, and/or the hash of the message body in order to prevent replay attacks, message tampering, and unauthorized requests from getting through.  HMAC does not need to rely on SSL, and therefore authentication can be provided over clear text without compromising the client or server.

Programming languages generally have built in functions to generate HMAC’s.


Although more complex to build and to implement, OAuth provides a lot of flexibility to your REST api authentication model.  OAuth2 allows a client to act on behalf of a user via access tokens. Clients are allowed to perform certain actions, and the actual user indicates which actions clients are allowed to perform by way of an authorization server.  Users need to confirm that they trust the client to do what the client wants to do.

OAuth has two flavours, version 1, and version 2.  Most new OAuth implementations use OAuth v2.

With OAuth, SSL/TLS is 100% required in order to maintain security and integrity.

REST Versioning

Unlike other technologies such as SOAP, there is no versioning specification for REST services.  Some organizations follow a URI versioning practice by explicitly using the version as part of the URI.


Some organizations do the opposite, where the /api/product will always reference the newest version, and old clients can choose to reference previous api versions if they want.

Other organizations will just make sure, if at all possible, that REST service updates are backwards compatible with the old services.  Adding new fields to the REST service should not break existing clients.

I’ve also seen some organizations use a query string parameter to specify the version.  I’m not too keen on this as I personally don’t like muddying up the URI with these kinds of query parameters, but you will see in these cases, REST URIs that look like /api/product?version=2



REST as Part of SOA

Service Orientated Architecture (SOA) has to do with service orientation. In the 2000’s and early 2010’s SOA had been all the rage, and many large organizations began adopting SOA and building SOAP (including WCF and other technology)  services as part of their SOA.

There are many different ways to create SOA and different patterns as well.  Imagine loosely coupled, business services that are packaged up and exposed and accessible over the Internet, intranet, or corporate extranet.  That’s basically SOA.  There is no standard SOA approach, and you can’t look at different systems following these principles and generally say that one is more SOA than the other as long as they are following this paradigm.

Although it’s not necessarily a rule of law, services generally will conform to best practices of software development and services will be autonomous, not cross service boundaries, and use contracts.

SOA doesn’t dictate which technology is in use which means, technically, many other technologies besides SOAP could be considered for SOA.  This brings us to REST.  Can RESTful become part of an SOA?  Yes, but there are some caveats and different schools of thought on the subject.  However, always remember that REST, in itself, is it’s own architectural style.

The architectural approach of a REST API follows established resource patterns around http and the world wide web as has been in place for 25 years.

Although not dictated, in typical SOA’s you may have an ESB (Enterprise Service Bus), that among other things, provides orchestration to your SOA services.  It may bundle up service calls in order to implement a full process called on the service bus.  This isn’t something native to REST Api’s, but there are orchestrator frameworks that work with REST services.

There are also some accepted principles of Service Orientated Architectures, and some of them are complimented well with REST whereas others do not.  Again, as per above, SOA is loosely defined, so acceptable principles can be thrown out the window and you can still call your architecture SOA, if you want.  Some of the accepted SOA principles that do not jive well with RESTful, include:

Service Discoverability – In SOA architectures, service discoverability is a key practice.  As an example, this can be done through WSDL which gives the client or client applications complete details about the service contract, version, and input and output parameters and types.  There is no such thing inherent in a REST service or REST Api.  REST follows standard web approaches with resources and methods (GET, PUT, POST, etc), and supplemental documentation is needed in order to document the API.  There are provisions that some Rest API’s use to provide contextual information about the REST service, but it is not standard to REST.

Standardized service contracts – Services adhere to a communications agreement, as defined collectively by one or more service-description documents.  Using SOAP within REST, for example, would expose the WSDL as the service contract.  This doesn’t exist for REST services,  The developer of a REST Api generally has a duty to ensure compatibility with old client, but unlike WSDL, SOAP, and other technologies typically associated with SOA, there is no standard approach to standardizing service contracts or versioning different versions.  See the section on REST versioning.

Because SOA, again, is loosely defined, you could create an SOA which includes REST services, and that can be considered reasonable.  However, unless their REST services are working in conjunction with a bunch of other services as part of an SOA, most organizations are referring to their REST architecture simply as REST Api architecture, or Web API architecture (for Microsoft technology), and not using the SOA terminology as frequently.



Post a Comment

Your email address will not be published. Required fields are marked *