REST API guidelines

REST API guidelines

REST APIs are fantastic because they leverage the power of the HTTP standard, providing a lightweight and efficient way to build robust web services. They offer built-in mechanisms for CRUD operations, caching, and more, aligning perfectly with HTTP methods and headers. However, challenges like filtering and pagination often require custom solutions. In this post, we'll explore how different REST API guidelines tackle these issues, comparing their approaches to help you find the best fit for your needs.

Filtering and sorting

Filtering data is crucial for creating efficient and user-friendly APIs, but it's also one of the least standardized aspects. Each guideline has its own take on how to handle filtering and sorting.

API Guideline Approach Example Details
JSON API Query parameters with filter prefix GET /articles?filter[author]=1 Uses the filter parameter for each filterable field, providing a structured and clear approach.
JSONAPI.net Query parameters with filter prefix GET /articles?filter=equals(author,'1') Uses the filter parameter with expressions.
GitHub REST API Query parameters GET /issues?labels=bug Provides resource-specific query parameters, allowing straightforward filtering.
Google API Design Guidelines Query parameters directly related to resource fields GET /books?author=Hemingway Uses direct field names as query parameters for intuitive filtering.
React-Admin Simple REST Data Provider Query parameters with JSON objects GET /posts?filter={"author":"John"} Allows complex filtering criteria by using JSON objects in query parameters.
Auth0 API Query parameters with q for queries GET /api/v2/users?q=email:"john.doe@example.com" Uses a q parameter for flexible and complex Lucene queries.

Analysis

  • Google and GitHub offer simple, easy-to-write filters, but may be limited for complex queries.
  • JSON API and React-Admin allow filtering of nested properties but might fall short for value comparisons.
  • JSONAPI.net balances readability and flexibility, though it can be complex to implement depending on the database.
  • Auth0 uses powerful Lucene queries, ideal for databases that support them, though it can complicate client-side code. As it exposes the database directly it creates a direct coupling to the data and may require parsing of the queries on the server.

Pagination

Effective pagination is key to handling large datasets without overwhelming the client or the server. Different guidelines have their own methods, ranging from cursor-based to page-based pagination.

API Guideline Approach Example Details
JSON API Page-based GET /articles?page[number]=2&page[size]=10 Includes first, prev, next, and last links in the response for easy navigation.
JSONAPI.net Query parameters with filter prefix GET /articles?page[number]=2&page[size]=10 Similar to JSON API, with structured pagination.
GitHub REST API Cursor-based GET /issues?labels=bug Uses Link headers in the response with URLs for next, prev, first, and last pages.
Google API Design Guidelines Cursor-based GET /books?pagetoken=abc123 Response includes a nextPageToken for retrieving the next set of results.
React-Admin Simple REST Data Provider Page-based GET /posts?page=2&perPage=10 Handles pagination with page and perPage query parameters.
Auth0 API Page-based GET /api/v2/users?page=1&per_page=10 Uses page and per_page query parameters to manage pagination.

Analysis

  • Cursor-based pagination (GitHub, Google) is better for large datasets and avoids issues with data changes.
  • Page-based pagination (JSON API, JSONAPI.net, React-Admin, Auth0) is straightforward for UI but can be inefficient for databases.

Response metadata

Metadata is often necessary for understanding the full context of the data, such as pagination details or the total number of items. Here’s how each guideline handles response metadata:

API Guideline Approach Example Details
JSON API Metadata in body json { "data": [ ... ], No standard for requesting metadata
JSONAPI.net Metadata in body json { "data": [ ... ], No standard for requesting metadata
GitHub REST API Link headers json [{}] Uses Link headers in the response with URLs for next, prev, first, and last pages.
Google API Design Guidelines Metadata in body json { "items": [ ... ], No standard for requesting metadata, but response always includes totalItems.
React-Admin Simple REST Data Provider Metadata in headers json [{}] Handles pagination with page and perPage query parameters.
Auth0 API Metadata in body json { "users": [ ... ], By specifying the includeTotals querystring the metadata in the body is enabled.

Analysis

  • JSON API, JSONAPI.net, Google, and Auth0 provide metadata in the response body, making it easy to access pagination info.
  • Auth0 makes it possible to explicitly request the totals metadata to be included in the response which is good from a performance perspective, but the response format changes which makes the client code more complex.
  • GitHub uses Link headers but does not natively include total item count.
  • React-Admin uses headers for metadata, leveraging standard HTTP features.

Common Fundamentals

When it comes to REST API design, there are several areas where most guidelines agree. These shared principles form the backbone of robust and intuitive APIs. Let's dive into some of these fundamental concepts and see how they harmonize across different guidelines.

HTTP Methods

First, let's talk about HTTP methods. These are the verbs of the web and map perfectly to CRUD (Create, Read, Update, Delete) operations.

Here’s a quick rundown of how they work, including their safety and idempotency:

  • GET: Used for retrieving resources. Safe, idempotent.
  • POST: Used for creating resources. Not safe, not idempotent.
  • PATCH: Used for updating properties of resources. Not safe, idempotent.
  • PUT: Used for replacing resources or collections of resources. Not safe, idempotent.
  • DELETE: Used for deleting resources. Not safe, idempotant.
  • OPTIONS: Used for retrieving the communication options for resources.
  • HEAD: Used for retrieving the headers for a resource, similar to GET but without the response body.

URI Conventions

Consistent URI design is another area of consensus. Here’s how it generally looks:

  • Collection: GET /articles
  • Single resource: GET /articles/{id}
  • Related resource: GET /articles/{id}/author

For actions that don’t fit neatly into CRUD operations, create specific endpoints using POST, as these actions typically change state and are not idempotent.

  • Example: POST /articles/{id}/publish

The casing of the resources are kebab-cased, such as blog-posts and the fields names are typically camel-cased, but github and auth0 are using snake-case.

The field names typically use prefixes to make it clearer that the field represents a boolean value:

  • isPrefix: Use is for fields that describe a state or condition. Example: isActive, isPublished, isVerified.

  • hasPrefix: Use has for fields that describe possession or presence. Example: hasAccess, hasPaid, hasLicense.

  • canPrefix: Use can for fields that describe capability. Example: canEdit, canDelete, canView.

The entities typically contain timestamps that seems to be consistent:

  • Created At: The timestamp when a resource was created.
    • Example: "createdAt": "2023-01-01T00:00:00Z"
  • Updated At: The timestamp when a resource was last updated.
    • Example: "updatedAt": "2023-01-02T12:34:56Z
  • Deleted At: The timestamp when a resource was deleted (for soft deletes).
    • Example: "deletedAt": "2023-01-03T08:45:23Z
  • Timestamp Formats: Use ISO 8601 format (e.g., "2023-01-01T00:00:00Z") for consistency and timezone awareness.

Response codes

Response codes are the feedback from your server, letting clients know how their request was handled. Here’s a quick guide to the most commonly used codes:

Success

  • 200 OK: Request was successful.
  • 201 Created: Resource was successfully created.
  • 202 Accepted: Request has been accepted for processing, but processing is not complete.
  • 204 No Content: Request was successful, but there is no content to return.

Client Errors

  • 400 Bad Request: Malformed request syntax or missing required parameters.
  • 401 Unauthorized: Authentication failed or missing.
  • 403 Forbidden: Authentication succeeded, but the user does not have permission.
  • 404 Not Found: Requested resource could not be found.
  • 405 Method Not Allowed: HTTP method not allowed for the resource.
  • 409 Conflict: Conflict with the current state of the resource.
  • 422 Unprocessable Entity: Well-formed request, but semantic errors.

Server Errors

  • 500 Internal Server Error: Server encountered an error and could not complete the request.
  • 503 Service Unavailable: Server is temporarily unable to handle the request.

Designing an API with an ElasticSearch backend

As ElasticSearch support Lucene queries the Auth0 approach of exposing a q or query param would work well. If the index contains more data than you want to expose you may need to truncate the Lucene query before sending it to the database.

As there are different use-cases for paging it would be ideal to support both page-based and cursor-based paging. ElasticSearch has a scroll API that could be used. Another option is to filter on the document id and pass the query for the next page as a base64 encoded object.

ElasticSearch returns the total count of matching documents in the query response so there's no performance benefit in making this optional. As a personal preference I prefer to use headers for passing response metadata and keeping the body as a simple array.