To explain why having something like a "JSON API" module in core makes sense, I first have to explain the process/choices that led to the current REST module in D8. I will explain later what "JSON API" is.
REST in D8: a short history
- 2012: Humble beginnings
Back in the days when Drupal 8 was in active development, we chose to add
rest.moduleto Drupal core, inspired by the https://www.drupal.org/project/restws contrib module in Drupal 7.
The scope of that issue is clear: it's minimal. It only does
DELETE, not yet
PATCH. It hardcodes a single format: JSON-LD (the up-and coming standard back then). It only supports content entities and one other resource (watchdog/dblog), to prove that it's possible. This is a fine starting point!
The question of whether to support collections, paging and queries is raised at, and is answered that
collections and paging are a problem for Views plugins, because Views is how we build collections— this also makes sense.
This was done in(committed November 2012).
- 2013: Default format: from JSON-LD to HAL+JSON
The default format was first JSON-LD. It was then decided to change that to HAL+JSON. Over time it had become clear that
shipping a perfect "primary format" was unnecessary, because Symfony's Serializer component makes it easy to add new formats. The primary use case for the REST module was expected to be content deployment. HAL was the simplest format with hypermedia support. Because of the involvement of JSON Schema proponents, HAL+JSON was chosen, because JSON Schema can be layered on top of HAL+JSON.
This was done in(committed March 2013).
- 2014: no significant changes (mostly updates for D8 API/best practice changes)
- 2015: no significant changes (mostly updates for D8 API/best practice changes)
- 2016: +
- The test coverage for the
halmodules was insufficient to guarantee correct behavior and good DX for users of D8's REST API. See the numerous issues listed in those two plan issues.
This is fine and understandable — all of the above was built by volunteers and limited sponsoring. Drupal 8's APIs changed a lot since 2012! The popularity of the HAL format didn't take off as expected.
REST in D8: fundamental pain points after one year
After one year of having Drupal 8 released, some pain points have become clear:
- Difficult to set up. From the extra layer of permissions (fixed by https://www.drupal.org/project/restui to make it simpler, it's more hassle than necessary for the 90% case: REST access to entities. ) to the complex configuration (simplified by ): it was far too hard. We've made it better in 8.2, but even with
- Collections: even harder to set up. For every single collection that you want to access, you need to create a separate view. If you need to access five entity types, you need to set up five views. As you can see in the history, this was intentional, but it leads to silly/frustrating situations. Like the very common case of having an article with comments, and it literally being impossible to
GETall comments for an article, because you need to set up a (relatively advanced) view for that. Not to mention that even when you set up a view to provide access a collection, you still don't have pagination: have 1 million nodes? You'll get all of them in a single response, except the request will time out.
- Easy to shoot yourself in the foot. Due to combinations of circumstances, the URL structures are confusing (
GET/PATCH/DELETE /node/42, but
POST /entity/node), forgetting a particular request header or an entity field results in incomprehensible error responses, and figuring out capabilities (such as embedding related resources) requires debugging the PHP code.
- Incomplete normalizations. This means you simply cannot access certain information. For example, translated entities, modifying config entities, file URLs, file uploads, image style URLs, parent term, entity paths, and so on. We are fixing those, which will also help the REST module and any other module providing RESTful web services.
We can fix #4, and doing so will help all modules providing RESTful web services.
The first 3 are fundamentally unsolvable, because that's how it was designed.
JSON API: the spec
JSON API is a specification: http://jsonapi.org/.
Like HAL, it describes a particular normalization. But on top of that, it also prescribes:
- how to handle collections: http://jsonapi.org/format/#fetching-resources
- how to handle filtering: http://jsonapi.org/format/#fetching-filtering
- how to handle sorting: http://jsonapi.org/format/#fetching-sorting
- how to handle relationships: http://jsonapi.org/format/#fetching-relationships + http://jsonapi.org/format/#crud-updating-relationships
- how to handle errors: http://jsonapi.org/format/#error-objects
- an extension system is in development: http://jsonapi.org/extensions/ — but the spec already ensures that it's possible to layer this on top: http://jsonapi.org/format/#content-negotiation
In other words: it is more restrictive, it defines much more of how a RESTful web service using it should work than the HAL specification (zero mentions of "collections".
Put differently: JSON API has a broader opinion, which removes the need for Drupal to have an opinion. This in turn allows developers to use non-Drupal-specific tools to consume Drupal 8's APIs.
JSON API: module
It made some choices:
- Only entities. Because only entities are guaranteed to have collections and relationships. And because it's the 99% use case.
- Rely on the Entity (Field) Access API. This means we can expose all entity types via JSON API by default: the responsibility lies with Entity API to ensure access is set up correctly — the rest of Drupal already relies on this too.
- All installed authentication providers are allowed.
- It doesn't expose the data structures that Drupal uses internally for entities + fields, unlike the REST module. See: https://www.youtube.com/embed/cDDSdIyjVkM
Together those decisions mean that installing the JSON API module means instant, format-standardized access to all entities in Drupal 8 without any configuration, via any of the available authentication providers.
Painless setup. Zero config (no need for documentation). A spec to look at to see what's possible (not Drupal-specific documentation).
As part of the API-first initiative (
e0ipso is the maintainer of the JSON API contrib module. It's not an early-stages module:
- It's been under development since May 2016.
- It's had a fair number of contributors.
- It has had weekly meetings to coordinate progress for the last several months.
- Its seen some usage already, despite little publicity and despite D8 core's REST: https://www.drupal.org/project/usage/jsonapi. That usage includes production sites.
- Wim Leers has done a comprehensive review to evaluate core readiness ( ), some improvements have already landed, and the most important architectural issue is being worked on right now ( ).
- It has received a 1.0-alpha3 release — if it weren't for #2829327, it'd have already been in beta stage.
We believe that JSON API may be a natural complement to the REST module. (As does Dries: http://buytaert.net/improving-drupal-8-api-first-json-api-oauth2).
The REST module will always have its own place, with its ability to support multiple formats, any URL structure, and custom REST resources. JSON API aims to make the 90% use case better.
The JSON API module completely solves the first 3 fundamental pain points of the REST module! The fourth pain point is one that is the same for both the REST module and the JSON API module — fixing it for one also fixes it for the other! In other words: a step forward on all fronts for a very common use case.
Note that JSON API is layering on top of the Entity API (exposing all its data), the serialization API (for its serialization needs), the routing API (for its routes) and the authentication provider API (to allow additional authentication providers to be used). That's why it's able to have zero config. And that's why it's easy to to have it as an experimental module: it can be installed and uninstalled at any time.
Build consensus that adding JSON API to core is a good idea.— see #18. Formulate detailed plan.— see — added in #20.
User interface changes
Data model changes