(Updated July 29, 2020)
This article provides design and architectural guidance for developing and consuming REST APIs starting with basic concepts and then going deeper into more advanced concepts.
REST Basics
REST is an open protocol for designing web based API’s and distributed applications not relying on any specific technology or vendor. Generally used over http and https, REST services use resources in order to provide data and functionality to clients typically in JSON format. Identifiers can be used along with resources in order to specifically identify a single item from a resource.
Since Rest APIs are platform independent by design there are generally no compatibility concerns. Typically a REST API will conform to a messaging format standard such as XML or JSON, however, there is no standard format that must be adhered to to create a Restful service. A restful service can return other types of data, and I’ve even seen CSV data returned from a restful service, although, that is something I’d condone except in extremely specific circumstances.
When thinking about a typical way to consume or interact with a rest API, imagine a list of products each with their own product id’s. The list of products will be accessible with a URI like:
https://mycompanyrestapi.com/api/products
Whereas the product with the specific product id of 12 will be directly accessible via the following:
https://mycompanyrestapi.com/api/products/12
These are both “GET” requests designed to retrieve data.
Resource Identification with Multiple Keys
As you see in the example in REST basics, a resource (item/record) is accessible through the resource URI. For example: https://mycompanyrestapi.com/api/products/12 will retrieve the product with the id of 12. This can be presented in different ways, and it does not have to use the identifier of the underlying database key, however, the identifier needs to be consistent across the entire api. Most Rest API implementations tend to use the backend key (such as database identifier) unless there are security risks in doing so.
There may be cases where a resource, in this example, a product, is not uniquely identifiable via a single key. This could be for a variety of reasons, including backend data storage architecture, or other storage or database constraints.
There are a few ways to approach this:
- Use a parameter filter to further filter and identify the actual product. Example: /api/products/keyboards?id=12 . – I generally don’t like this approach and I don’t think it’s very useable. /api/products/{id} will no longer return a single resource, and instead it will return a list of resources that need to be filtered further with a parameter to get the product of keyboard category with the identifier of 12.
- A better approach to above, and the more rest compliant approach, would be to create a routing or a url path which includes the product category in the URI. So, your URI would be formed as follows: /api/productcategories/{categoryid}/products/{productid} . Essentially, rest uri’s can be designed in anyway that makes sense to you as long as the hierarchy makes sense.
- If product id truly uses a 2 part identifier, but you can’t find a use case to use the approach above, you can consider concatenating your 2 part key in order to create a unique key, or find some other combination of values that guarantee that the id is unique for the resource. This means your identifier used by rest clients will not necessarily match the true identifier in the backend storage of your systems, so you will need to make sure that the same unique identifier is used across your RESTful services which use or refer to the resource in question. With the product example, you would then have a Uri with an id such as: /api/products/keyboard-12
I’ve run into this situation quite a few times. We had a medium sized Azure implementation in which I came on board and lead the architecture for. Because the design decision based on costing was to use Azure tables for some of our storage needs, we designed a list of resources with the standard Partition Key and Row Key in Azure, where the two properties uniquely identify an item. When creating a REST Api around our Azure data infrastructure, I felt it was appropriate to use a combined key of the Partition Key and Row Key to identify the resource in the REST Api (number 3 above).
In another case, the data entities and resources were much bigger and much more complex with many more client use cases. It was an enterprise implementation, and therefore due to requirements, ease of use, and ease of maintenance, I designed the Rest API using nested resources (number 2 above). The great thing about this approach is that the nesting can take multiple forms. You can have the product resource nested directly under category, and also nested directly under another resource, distributer, for example. So you end up with multiple nesting options for clients to access the data they need. For example: /api/categories/{categoryId}/products/{productId} will return of specific product. /api/distributer/{distributerId}/products/ will list all products for a given distributer and /api/distributer/{distributerId}/category/{categoryId}/products/ will list a all products for a given distributer in a given category.
Use Nouns to Identify Resources
Your Rest API should be say to use and navigate, and one of the standard pillars of that is to ensure your API can be navigated through a set of different resources. Example: Use /api/products and /api/products/{0} to get a list of products or a single product. When updating a product or creating a new product you will also use the same resource names but instead of the HTTP Method being “GET” it would be “POST” or “PUT”.
This means you should not use a verb as part of your REST API identification. URI’s such as the following are a no-no: /api/getproduct or /api/newproduct.
Relations to Other Resources
Although there is no JSON standard for linking one resource to another, there are some generally approaches used in REST APIs. When dealing with REST data, relationships will exist between one resource and another, and this relationship is generally signified using a URI as we see in the sample JSON below for product id 12. There are different approaches, but quite often a links array will be included as part of the JSON data which will identify the link and provide the URI to that specific links related resources.
{
"productID":12,
"name":"A really cool stereo",
"price":399.99,
"links": [
{"rel":"distributors","href":"https://mycompanyrestapi.com/api/distributors/350", "action":"GET" },
{"rel":"vendors","href":"https://mycompanyrestapi.com/vendors/178", "action":"GET" }
]
}
A lot is assumed with this JSON data example. For example, it assumes that there is only one distributor, but in the real world there could be multiple distributors and therefor the JSON could be returned in a variety of different ways depending on how the creator of the REST service designed it. The JSON data for a product could have the basic entities for each distributor, indicating the name of the distributor along with the Id and link to the distributors URI for that distributor such as in the following example.
{ "productID":12, "name":"A really cool stereo", "price":399.99, "distributors": [{"id":350, "name":"SuperLight Distributors", "href": "https://mycompanyrestapi.com/api/distributors/350"},{"id":357, "name":"Heavytown Distributors", "href": "https://mycompanyrestapi.com/api/distributors/357"}, ] }
Notice that there are some differences here in the generally approach taken to linking to other resources. There is no standard approach to linking with JSON data, so each RESTful service will generally follow established patterns and ensure they are documented in the REST Api documentation. Even in these two examples, there are differences in how the Uri’s are provided, both structurally within the JSON and the amount of data included. The second example, the client will assume that it is a GET request whereas in the first example, “GET” is explicitly indicated.
Stateless Architecture
There are no sessions or session states in a RESTful service or architecture. The client is solely responsible for managing state if necessary, and is solely responsible for providing all the needed data to the REST service in order for the request to be successfully fulfilled.
Securing Your REST Services
REST services should not store any authenticated data from the client, and although some REST Api’s do maintain authentication state for a client such as through a cookie or header token, it is a bad practice. Generally, every REST request should contain the authentication data the service needs to validate authentication prior to executing.
Most REST services will need to be secured. The first step in this process is authentication and authorization.
Authenticating and Authorizing a REST request
When a connection to a REST service is made, the credentials supplied to it are verified to confirm that the client making the call is verified to be who they claim to be. Now that the client has been authenticated, they need to be authorized. Authorization just determines if they have specific access or authentication to the resource they are trying to access. As a best practice, securing the communication between clients (via SSL/TLS) and rest services is something that should be done, it’s even more important and mandatory when using Basic Authentication, OAuth, or another authentication scheme where the authentication credentials can be accessed in plain text.
Types of Authentication
Authentication for REST services can be handled in a variety of ways. Here is a list of some of the more common approached to authentication.
HTTP Basic Authentication: This authentication mechanism is simple, and is part of the HTTP standard. The implementation is just to provide a username and password in plain text as part of the HTTP header of the request. The name of the header is “Authorization”, and the user name and password are provided in base64 encoding with a (:) separating the username and password (username:password). Obviously, this is only secure if the communication is encrypted (SSL). The response status from the server will either be 200 – OK, or 401 – Unauthorized. With HTTP Basic Authentication. the username and password are sent from the client to the server on every request, and the server authenticates the client on every request.
Hash Based Method Authentication Code (HMAC):
In an HMAC authentication scenario, the security lies around a secret key that is known by both the server and client. This key can be used to verify both the data integrity of the message as well as to authenticate. The HMAC is calculated with a cryptographic hash function such as MD5 or SHA-1.
The components involved for HMAC authentication are the cryptographic hash function, the secret key, and the message to be authenticated, and an internal key which is derived from the original key.
To generate an HMAC, there are two hash computations involved, an internal and external. The internal hash is produced first, and then the external hash is produced deriving from the internal hash combined with the inner key.
With HMAC, integrity of the message is kept in check, by including the timestamp, a random number, and/or the hash of the message body in order to prevent replay attacks, message tampering, and unauthorized requests from getting through. HMAC does not need to rely on SSL, and therefore authentication can be provided over clear text without compromising the client or server.
Programming languages generally have built in functions to generate HMAC’s.
OAuth2:
Although more complex to build and to implement, OAuth provides a lot of flexibility to your REST api authentication model. OAuth2 allows a client to act on behalf of a user via access tokens. Clients are allowed to perform certain actions, and the actual user indicates which actions clients are allowed to perform by way of an authorization server. Users need to confirm that they trust the client to do what the client wants to do.
OAuth has two flavours, version 1, and version 2. Most new OAuth implementations use OAuth v2.
With OAuth, SSL/TLS is 100% required in order to maintain security and integrity.
REST Versioning
Unlike other technologies such as SOAP, there is no versioning specification for REST services. Some organizations follow a URI versioning practice by explicitly using the version as part of the URI.
URL Versioning
https://mycompanyrestapi.com/api/v2/product
https://mycompanyrestapi.com/api/product
Some organizations do the opposite, where the /api/product will always reference the newest version, and old clients can choose to reference previous api versions if they want. This can be accomplished fairly easily with an API Management layer in front of your services which will always direct to the latest compatible version of the API internally.
Query String Versioning
Other organizations will just make sure, if at all possible, that REST service updates are backwards compatible with the old services. Adding new fields to the REST service should not break existing clients.
I’ve also seen some organizations use a query string parameter to specify the version. I’m not too keen on this as I personally don’t like muddying up the URI with these kinds of query parameters, but you will see in these cases, REST URIs that look like /api/product?version=2
Header Versioning
Header versioning requires a custom header provided as part of the HTTP request. The service would read the header and know which API the requester has intended (this can even be a function of API Management) and act accordingly. This is similar to query string versioning from a client implementation perspective except that the versioning is much more obscured as it is provided as part of the header. This could add implementation complexity in order for the client to add custom headers to their REST API requests.
REST Operations
Methods are part of the HTTP standard and are also used to call REST APIs and must be considered as part of a REST API design.
Your typical operations for REST API services, include:
-
- GET: Used to get one or more resources.
- POST: Create a new resource
- PUT: Update an existing resource, and if the service supports it, the new resource is created if it didn’t already exist.
- DELETE: Delete an existing resource
- PATCH: The least supported of typical REST API designs, however PATCH is quite useful in updating a resource partially
REST as Part of SOA
Service Orientated Architecture (SOA) has to do with service orientation. In the 2000’s and early 2010’s SOA had been all the rage, and many large organizations began adopting SOA and building SOAP (including WCF and other technology) services as part of their SOA.
There are many different ways to create SOA and different patterns as well. Imagine loosely coupled, business services that are packaged up and exposed and accessible over the Internet, intranet, or corporate extranet. That’s basically SOA. There is no standard SOA approach, and you can’t look at different systems following these principles and generally say that one is more SOA than the other as long as they are following this paradigm.
Although it’s not necessarily a rule of law, services generally will conform to best practices of software development and services will be autonomous, not cross service boundaries, and use contracts.
SOA doesn’t dictate which technology is in use which means, technically, many other technologies besides SOAP could be considered for SOA. This brings us to REST. Can RESTful become part of an SOA? Yes, but there are some caveats and different schools of thought on the subject. However, always remember that REST, in itself, is it’s own architectural style.
The architectural approach of a REST API follows established resource patterns around http and the world wide web as has been in place for 25 years.
Although not dictated, in typical SOA’s you may have an ESB (Enterprise Service Bus), that among other things, provides orchestration to your SOA services. It may bundle up service calls in order to implement a full process called on the service bus. This isn’t something native to REST Api’s, but there are orchestrator frameworks that work with REST services.
There are also some accepted principles of Service Orientated Architectures, and some of them are complimented well with REST whereas others do not. Again, as per above, SOA is loosely defined, so acceptable principles can be thrown out the window and you can still call your architecture SOA, if you want. Some of the accepted SOA principles that do not jive well with RESTful, include:
Service Discoverability – In SOA architectures, service discoverability is a key practice. As an example, this can be done through WSDL which gives the client or client applications complete details about the service contract, version, and input and output parameters and types. There is no such thing inherent in a REST service or REST Api. REST follows standard web approaches with resources and methods (GET, PUT, POST, etc), and supplemental documentation is needed in order to document the API. There are provisions that some Rest API’s use to provide contextual information about the REST service, but it is not standard to REST.
Standardized service contracts – Services adhere to a communications agreement, as defined collectively by one or more service-description documents. Using SOAP within REST, for example, would expose the WSDL as the service contract. This doesn’t exist for REST services, The developer of a REST Api generally has a duty to ensure compatibility with old client, but unlike WSDL, SOAP, and other technologies typically associated with SOA, there is no standard approach to standardizing service contracts or versioning different versions. See the section on REST versioning.
Because SOA, again, is loosely defined, you could create an SOA which includes REST services, and that can be considered reasonable. However, unless their REST services are working in conjunction with a bunch of other services as part of an SOA, most organizations are referring to their REST architecture simply as REST Api architecture, or Web API architecture (for Microsoft technology), and not using the SOA terminology as frequently.