[META] Start creating the public PHP API of the JSON:API module

Thanks for creating this issue. Rather than doing what Drupal has traditionally done and provide APIs for all the things, the JSON:API module takes the approach that many other software platforms take: keep APIs private and unsupported (@internal in Drupal parlance) until they've been sufficiently validated by developers building on top of it.

I'm relieved we did it this way, otherwise the JSON:API module would not have been in nearly as good shape. It has evolved a lot in the last 18 months.

So, let's wait and see which public APIs contrib module developers request. The sequence:

1. multiple developers raise a certain need
2. multiple developers validate the structure of an @internal API — iterate based on feedback
3. make API public (remove @internal)

That's fair :) I meant #2 to be the rule of thumb, not as an iron-clad process. So 👍

_{P.S.: for the particular use case that you're citing — field aliasing — I'm actually not convinced that we need a public API for that. I think we could and should solve this at the Typed Data level, much like https://www.drupal.org/node/2916592. That would then could then also serve as the facility for Drupal entity types to rename fields while retaining BC.}

The JSON:API Comment module exposes routes that enhance the experience of using JSON:API with comments.

It is forced to use several of JSON:API's internals to accomplish this. Primarily, the Link and LinkCollection classes and several methods in EntityResource.php.

WRT to links, I think #3014704: Expose API to allow links handling for entities from other modules describes the need (at least its title does). There's an example of how we might accomplish that in the JSON:API Hypermedia module. That module decorates the link collection normalizer and then passes the link collection to be normalized into a "LinkProvider" which is able to add/remove/update any links. This pattern worked well and I think could be imported into core JSON:API.

The primary shortcoming I found was links is that it is difficult to handle access + cacheability. Drupal\Core\Url has an access method which I think is meant to be used to hide links when the target resource is inaccessible, but it doesn't return cacheability information. IOW, it's hard to associate cacheability information with the response if a link is not in the response. I think that the Link class could gain a new argument to its constructor which would take an AccessResult. The Link would then be omitted if access is forbidden but its cacheability could still be added to the response.

As for EntityResource, the JSON:API Comment module extends it for two purposes:

1. to provide a custom collection
2. to accept POST requests and add some default field data

The custom collection is fairly standalone. Its primary dependencies are on the classes in the JsonApiResource namespace. I found no shortcomings with the API for those objects. After that, it used EntityResource::getIncludes and EntityResource::buildWrappedResponse. I think that shows that the IncludeResolver will likely need to be public. OTOH, buildWrappedResponse doesn't provide much value. I just used it because I could. I don't think it needs to be a public API. Finally, there was a dependency on EntityAccessChecker::getAccessCheckedResourceObject; again, I found no real shortcoming there.

Accepting POST requests is a different story. My implementation is very tightly coupled to EntityResource because I was forced to copy the contents of createIndividual to do deserialize, check access, validate and return a compliant response. This was only necessary because I needed to add some default field values to the parsed entity. If I had a more granular way to do that, my code could have been much cleaner.

Finally, what I discovered that is not closely related to concrete code is that custom JSON:API resource may not be able to support the sort, filter and page query parameters because those are closely coupled to an entity query and your custom collection may not necessarily be backed by something that can be made into an entity query. If we provide an API for custom collections, we should make sure that those come à la carte. OTOH, it seems that the fields and include parameters may be broadly applicable.

I created #3055889: JsonApiResource\Link objects with inaccessible target urls should not be normalized based on #6.

#6: Exciting! :) Thanks for sharing your insights!

The primary shortcoming I found was links is that it is difficult to handle access + cacheability. Drupal\Core\Url has an access method which I think is meant to be used to hide links when the target resource is inaccessible, but it doesn't return cacheability information.

Indeed. Instead of working around this limitation, let's fix it? This is now a core module, so we can fix core :) This can use the same backwards-compatible pattern as \Drupal\Core\Access\AccessibleInterface::access().

add some default field data

Can't we fix this in core too? i.e. at the Entity/Field/Typed Data level?

Indeed. Instead of working around this limitation, let's fix it? This is now a core module, so we can fix core :) This can use the same backwards-compatible pattern as \Drupal\Core\Access\AccessibleInterface::access().

I think we can/should do that. I don't know if it would completely solve the problem though. I think that only checks the access requirements on a route. At present, we have certain access controls run in controllers. Also, consider this link which I added as part of my recent project:

{
  "unpublish": {
    "href": "http://drupal.test/jsonapi/comment/comment/4cee0abb-7e9e-4c85-8b71-8557008efc84",
      "meta": {
        "linkParams": {
          "rel": [
            "https://jsonapi.org/profiles/drupal/hypermedia/#update"
          ],
          "data": {
            "type": "comment--comment",
            "id": "4cee0abb-7e9e-4c85-8b71-8557008efc84",
            "attributes": {
              "status": 0
            }
          },
          "title": "Unpublish"
        }
      }
  }
}

Route access would only indicate whether the user has update permission for the comment, which could be true even if the user does not have the administer comments permission. So, you'd want to hide this link based on more specific knowledge that a route access checker could provide.

Can't we fix this in core too? i.e. at the Entity/Field/Typed Data level?

I'm not sure TBH. It's a weird one. The field values need to be set prior to entity validation and they depend on the current route. In my case, I'd only want those default values to be set on a certain route, not for all entity save operations. I think what would be more effective in this case might be pre and post-(de)normalization hooks of some kind.

That's something we might want to fix, but how is it relevant to this issue?

TBC, I actually don't think we want to rely on Url::access. I brought it up because a Url object is the second argument to Link. I was trying to anticipate the question, "why isn't Url::access totally sufficient and why can't we just make that better?" The answer is: "because route access alone is not enough, you may want to run more granular/extra access checks". I guess I didn't do a very good job at communicating that.

Why would we want to invoke Url::access() in the case where we know we're operating on an entity and so can invoke the entity's access methods? Are there other cases you have in mind where JSON:API needs to perform access on something that isn't an entity?

When I said: "I think that the Link class could gain a new argument to its constructor which would take an AccessResult. I was imagining an API that would be used kind of like this:

$update_access = $entity->access('update', NULL, TRUE);
$status_access = $entity->{$status_field_name}->access('edit', NULL, TRUE);
$can_publish = AccessResult::allowedIf(!(bool) $entity->status->value);
$can_unpublish = AccessResult::allowedIf((bool) $entity->status->value);
$update_status_access = $update_access->andIf($status_access);
$url = $entity->getUrl('jsonapi');
$publish_link = new Link(
  $cacheability,
  $url,
  $link_rels,
  $attribs,
  $update_status_access->andIf($can_publish)
);
$unpublish_link = new Link(
  $cacheability,
  $url,
  $link_rels,
  $attribs,
  $update_status_access->andIf($can_unpublish)
);
return $link_collection->withLink('publish', $publish_link)->withLink('unpublish', $unpublish_link);

@effulgentsia, I think that's probably pretty similar to where your head was at. You can see that it's running entity/field access checks, but it's also running an extra check on the field value itself. That's there to drive the client-side application state a bit. IOW, we only show a link that makes sense at the given moment, not just whether its strictly allowed or not.

Hello,

Thanks for opening this issue.

As the maintainer of Entity share https://www.drupal.org/project/entity_share, here are my needs:

I have a service (https://git.drupalcode.org/project/entity_share/blob/8.x-2.x/modules/ent...), entity_share_client.jsonapi_helper (which will need to be splitted on the future...) which needs 3 internal services from JSON:API:

• jsonapi.serializer
• serializer.normalizer.jsonapi_document_toplevel.jsonapi
• jsonapi.resource_type.repository

jsonapi.serializer is only used to be passed to serializer.normalizer.jsonapi_document_toplevel.jsonapi.

serializer.normalizer.jsonapi_document_toplevel.jsonapi is used to get an entity from JSON data.

jsonapi.resource_type.repository is used for serializer.normalizer.jsonapi_document_toplevel.jsonapi and to handle field alias (getInternalName() and getPublicName()). I also think to use it for the hasField() method in the future to check data structure.

Last year, there has been a discussion to be able to not rely on those services #2939827: Provide a supported API for entity denormalization and a possible solution #2939827-19: Provide a supported API for entity denormalization would be to use \Symfony\Component\HttpKernel\HttpKernelInterface::SUB_REQUEST.

I only tried one time to use this solution, unfortunately since the beginning of 2018, I am barely assigned time to Entity share, I hope to have more time (and more important, in a regular way) stating from now.

Now that there are more modules in the JSON:API ecosystem (https://www.drupal.org/project/jsonapi/ecosystem) and also https://www.drupal.org/project/jsonapi_operations. Maybe I will find more examples.

Also, initialy the issue #2939827: Provide a supported API for entity denormalization had been created in the JSON:API issue queue. Should it be back there?

Thank you all for your work on Drupal and JSON:API!

Hope to see you at DrupalDevDays 2019!

Moving to Drupal core's issue queue.

#3037039: Create a public API for indicating resource types should not be exposed just got committed and is the first public PHP API in the jsonapi module! 🎉

The second one is already on its way: #3085035: Add a public API for aliasing and disabling JSON:API resource type fields builds on that first one and is also RTBC 😀

Hi

One more use case is sorting comments by thread.

1. With JSONAPI we can get comments of exact node and sort them by thread:
/jsonapi/comment/comment?filter[by_node][condition][path]=entity_id.id&filter[by_node][condition][value]=node_uuid_here&sort=thread

2. But the result is not we want to see, because the "ORDER BY" is a little bit more complicated: https://github.com/drupal/core/blob/8.8.x/modules/comment/src/CommentSto...

3. Comments module provides custom sorting for views: https://github.com/drupal/core/blob/8.8.x/modules/comment/src/Plugin/vie...

4. It would be nice to have the same approach for JSONAPI: customize sorting by defining the plugin in 3rd party module

Have you seen the JSON:API Comment module yet @dmitry.kazberovich? That module creates new JSON:API endpoints to fetching, authoring and replying to comments. It also takes care of thread sorting. Additionally, the Fluid Comment module contains a React app that supports comment threading based on that former module if you need examples.

#3085035: Add a public API for aliasing and disabling JSON:API resource type fields was also completed! 🥳

New: #3100732: Allow specifying metadata on JSON:API objects.

I know the component is "jsonapi.module", but I'm giving this a more self-documenting title for when it shows up in issue listings across core or per-user. ;)

Cheers,
-Derek

👍 Thanks!

I would like to see a way for entity types to specify the resource type ID in their annotations.

Take the commerce_product entity type. None of our URLs include "commerce_product", they're all "product/". I would expect to be able to have the same control over JSON API links, so that I can specify "products" and have the URL be "jsonapi/products/clothing", etc.

This is currently doable by decorating the resource type repository and providing a custom resource type object (with an overridden getPath() method), but that is a bit verbose for what feels like a common need.

#21: created an issue for that: #3105318: Add a public API for aliasing resource type names.

Probably another one for in this list: #3104408: Allow to include a total count of items to collection responses

Added documentation issue for ResourceTypeBuildEvent since that has not been done in any of the issues yet.

Moving issue to shipped since it has been committed for 9.3

Adding #3026432: Increase max pager size, ideally per resource type after moving it from the contrib jsonapi project from ~2.5 years ago into the Drupal core issue queue.

Indicating which Drupal minor each public PHP API shipped in 😊

Further to #3100732: Allow specifying metadata on JSON:API objects, there doesn't seem to be a method to inject a meta member (https://jsonapi.org/format/#document-meta) at the top level of an API response. Should there be a new issue to address that? My use case is to expose overall content licence and site api version data, independent of the jsonapi version.

Yeah that should be a new issue.

Not sure if this is the right place for this (please point me in the right direction if there is a more useful place), but just thought I'd drop in a few things that I'm struggling with while developing an API over the last few weeks. These mainly revolving around the extensibility of core JSON:API services due to method access modifiers (predominantly the EntityResource controller class).

What lead me here is I wanted to try and re-use the core code in EntityResource::getJsonApiParams() and EntityResource::getCollectionQuery() in a Custom Resource by calling these methods, but couldn't because they have protected access, I then went down a rabbit hole...

For context I'm creating a few Custom endpoints using the JSON:API Resource module with JSON:API Extras installed to enable me to easily alias resources and provide defaults, and JSON:API Include allowing me to provide a cleaner response for the consumer in certain scenarios. Quite the module cocktail!

All of these modules are great and have worked OOB. However a lot of the time they have needed to modify a core JSON:API service, add a "shim" service or create a new service that extend an existing one etc. The main reason for this is to allow them to extend protected methods on the base service. Trying to build on-top of this very tricky when developing a module to extend the core features as you now have to be aware that popular modules (I imagine JSON:API Extras is pretty much a standard) are rolling their own instances of core services so you cannot reliably extend them yourself.

For example jsonapi_defaults part of JSON:API Extras replaces the class used for the jsonapi.entity_resource allowing them to add default parameters in the protected getJsonApiParams method. Had the getJsonApiParams been public the same could have been achieved with a service decorator, which would then allow other modules to add decorators as well without having to know what other modules are trying to extend these services. As it stands, if I want to extend some functionality to `getJsonApiParams` and retain the functionality provided by jsonapi_defaults, I have to include the jsonapi_defaults module as a dependency and extend that class.

As the module ecosystem around JSON:API grows, this is going to a bit of a blocker. Apologies if I don't have the back-story to the method access on these services (I'm sure there is a good reason)!

My two cents (for what it's worth) on things that might help the DX around extending JSON:API:

1. Merge functionality offered by JSON:API Extras into core?
2. Open up method access on core services. Allowing useful things like:
   1. $params = \Drupal::service(Routes::CONTROLLER_SERVICE_NAME)->getJsonApiParams(Request $request, $resource_type);
   2. $query = \Drupal::service(Routes::CONTROLLER_SERVICE_NAME)->getCollectionQuery($resource_type, $params, $cacheability);
3. EntityResource controller is doing too much IMO
   1. Filter parsing logic could be broken out into a separate public service? This could then be easily extended by decorators.
   2. Query building logic could be broken out into a separate public service? This could then be easily extended by decorators.
   3. Response building logic could be broken out into a separate public service? This could then be easily extended by decorators.
   4. Triggering some events during the execution of all of the above steps might also be an easier way to open it up slightly?

Hopefully that's a useful view of working with the module and it's ecosystem (which is ace btw) and might help a bit with moving it forward. For now I'll build my new features specifically for the application in question as I know what the dependency tree is and can extend the relevant classes.

Cheers! 🤟

#3100732: Allow specifying metadata on JSON:API objects is in!