[META] The Last Link: A Hypermedia Story

[I'm] unclear what the actual use of those urls are. I would argue that a well built api should be usable by a human (developer) as well. The complicated urls make it way harder to understand whats going on there.

I'm 100% with you on the API being "human-friendly". These aren't URLs they're URIs (i vs L). IOW, they're "Universal Resource Identifiers" and what they are is machine-readable, unique strings that identify codified meanings. Let'ts break one down...

drupal://jsonapi/extensions/commerce/links/relation-types/add-to-cart

This identifies a link relation type unique to Drupal, used by JSON API, but provided by the commerce module. Its name is "add-to-cart". If we had these, we would have a mechanism to discover and document the meaning of this link relation type. Its meaning would be "a link of this relation type is a link that tells the client how to add a resource to a user's cart". In my example, it would have a "human-friendly" alias of add-to-cart. Let's try another...

drupal://jsonapi/links/target-attributes/operation

This is a target attribute unique to Drupal, used by JSON API. It describes the link as one that provides an "operation" that the client can perform with the link. drupal://jsonapi/links/target-attributes/operations/add-to-relationship identifies that operation. That is, it denotes the procedure the client should perform. In this case, that the client should send a POST request to the given href using the context resource ({"type": "post", "id": 1}). These operations would be documented and could be "built in" with the JS client.

I just opened #2994211: [Meta] Expose Form Metadata via Schemata: [Meta] Expose Form Metadata via Schemata to track this concept

See also: https://github.com/jsdrupal/drupal-admin-ui/issues/262

I would want a client which doesn't depend on anything custom in our api.... Then as a result the client code should be able to work with any api service which provides the required information (such as OpenAPI spec for JSON API and the links to the api's extensions).

Amen. This is probably the lowest priority thing for me at the moment though. I don't think a client can be built that has all those automations until we've formalized the server side of things.

Renamed every instance of action to operation.

Added a child issue.

I'm sorry it's taking me so long to find the time to read this.

Don't be sorry, it's FOSS after all :)

I know I align with most principles that Gabe has shared about this in the past. I have been trying to timidly promote them myself, so I have confidence I'll be in line with most of this.

❤️

I'd like to float the idea to move this to a separate contrib.

I see a lot of space here for other modules and so I guess we probably will agree on that sentiment.

Certain parts of this must live here though, like the core idea of adding/removing links from different places in the document in a thoughtful, spec-compliant and BC-safe way.

In my example, I showed an "add-to-cart" link. As I envision it, that link would not be generated by JSON API, it would be generated by something like a commerce_jsonapi or my_store_customizations module. To get that link into the response, that module would implement an event subscriber. And JSON API would be the one to fire the event. So, it would be JSON API's responsibility to aggregate and validate all those 3rd party links, but not to figure out which links to add/not add.

For instance, I'd like JSON API to ensure that any link relation types are in fact valid ones. Or, if two 3rd party modules provide links with the same URI but different link relation types, I'd like to merge those two links into one link with both relation types on it.

Finally, certain core related links should also probably live in JSON API. Just like we generate the self link for all resources. This issue might mean that we expand on that a bit and add the edit link relation type to the self link if the user has permission to update that entity, etc.

In fact, I think we should start developing this in JSON API Extras (or JSON API Links) and then evaluate if we want to move this to JSON API. I see no strong reason against this.

I'm fine with prototyping much of this in a separate contrib. However, that raises the question, what internal API can it extend? I think that's the must I was referring to. I think the "right" API for that is an event subscriber used internally.

I would prefer not to put this in Extras because I fear it's becoming the equivalent of REST UI in that it's practically a necessity (perhaps only perceived) rather than a pure enhancement or proving ground.

I don't see it's a problem that JSON API Extras is very relevant. In fact I see it as a great sign. I agree that Extras is essential to any jsonapi site.

I agree with all of this :) It is not a problem that JSON API Extras is very relevant. Full stop.

What I think is a problem is if JSON API is not useful without Extras, and it's essentially an implicit dependency.

How does that impact this issue?

Let's say that the Admin UI Initiative wants to to show an "Edit" button on the content overview page and that the button must only be shown if hook_entity_access('update', $entity) is allowed. It would be poor UX to show that button and when a user clicks it, to then go to a forbidden page (or worse yet, show an error on save). How can they communicate the access results of that hook to the frontend in an elegant way?

The answer is of course hypermedia controls. Without them, they'll need to replicate access logic on the frontend or come up with some workaround that couples their implementation tightly to the backend.

Eventually, when their work is added to core, they'll only be able to rely on what's in core, where JSON API proposes to be. They won't be able to rely on JSON API Extras for that and so JSON API won't be useful to them in that scenario.

But I agree with @e0ipso's main point: the bulk of the linkage functionality (the API that allows other modules like Drupal Commerce to add additional links) ought not to live in the JSON API module.

@e0ipso and I came to a compromise in chat, which I hope you'll also agree with. JSON API will have classes for "link collections" (the parallel to JSON APIs "links object") and a "web link" (a value object for a link mirroring RFC8288's conception of a link). The normalizer for the link collection can then be overridden in a separate contrib which provides the extension points that something like Drupal Commerce could use.

#16

What if two contrib want to implement this? Or what if Drupal Commerce provides one for "core", and Shipping needs to provide way? Does this scale?

@mglaman, I think you may have missed a subtle point:

The normalizer for the link collection can then be overridden in a separate contrib which provides the extension points that something like Drupal Commerce could use.

I wasn't suggesting that modules all override the same normalizer. I was thinking that a module named jsonapi_hypermedia would override the normalizer once. From there, the jsonapi_hypermedia would dispatch events that 3rd party modules would subscribe to (like Commerce for example).

To help mitigate performance concerns, the event names could be specific to particular parts of the document and the events could carry a lot of contextual information so that subscribers can do as little processing as possible before escaping when they don't need to add links.

"Marking up" a document with solid linking will always have some trade-off between response size+speed and usefulness. It's up to the module maintainer adding links to be conscious of cacheing and it's up to the site builder to be conscious about what gets enabled.

#17:

It's not lost on me that adding support for hypermedia as the engine of application state means we'll be introducing stateful links ;). It's kind of tautological really.

I'm not sure what to conclude from that comment. I think you just mean that we should be mindful of this?

My thinking has always been that links need to be able to carry cacheable metadata, link providers need to be considerate of caching, and links should be optional feature of the response document.

Toggling links would be an excellent use case for negotiable profiles (they just landed in JSON API 1.1 last week). But before that, I think enabling or disabling jsonapi_hypermedia should be sufficient to get started with this.

In any case, I don't think performance concerns are a reason to postpone or hesitate on this feature.

In the add-to-cart example in the issue summary, how would the client know which HTTP method to use and what the appropriate body would be?

This is why my proposal is mostly about link relations. It's the link relations that define the semantics of the link, which can encompass everything from the which methods are acceptable to what data to send.

Since I wrote the issue summary, my thinking has changed a little bit and the example is outdated.

I see the link being more like this:

{
  "type": "product",
  "id": "1",
  "links": {
    "add-to-cart": {
      "href": "https://example.com/order/4/relationships/items",
      "rel": [
        "https://jsonapi.org/profiles/drupal/actions/#add-to-relationship",
        "drupal://jsonapi/extensions/commerce/links/relation-types/add-to-cart/"
      ],
      "params": {
        "arity": 3,
      }
    }
  }
}

The above is using my JSON:API links proposal.

We see that add-to-cart is no longer a link relation, just a string. Instead, the link has two link relations:

1. https://jsonapi.org/profiles/drupal/actions/#add-to-relationship
2. drupal://jsonapi/extensions/commerce/links/relation-types/add-to-cart/

The first, is a link relation defined by a yet-to-be-written profile. It would be defined like so:

This profile establishes the extension link relation https://jsonapi.org/profiles/drupal/actions/#add-to-relationship (henceforth, only written as add-to-relationship for readability). A client application processing an add-to-relationship link MUST use the HTTP POST verb when issuing any requests to the link's href URI. Additionally, the client MUST send a request document containing only the resource identifier object for the context object—or the resource identifier object for the object identified by the link's anchor parameter, if one is defined—as its primary data.

I didn't mention arity, just to keep it simple here.

The drupal://jsonapi/extensions/commerce/links/relation-types/add-to-cart/ relation would not be defined by a profile, but would reside in a namespace that lets any Drupal module define its own relations. We could define a docs page for these. Essentially, the add-to-cart would just "piggy-back" on the add-to-relationship relation. It's just there to disambiguate this particular add-to-relationship link from another one. It says, this link would add an item to a users cart. Maybe there would also be an add-to-wishlist link with exactly the same operational semantics (POST, etc.) but which has different meaning for this e-commerce site.

But what about state changes that do not involve modifying connections between resources, but that are effectively aliases for PATCH requests to the current resource, to improve developer ergonomics?

You're spot on! I think the same actions profile could define a new link relation, https://jsonapi.org/profiles/drupal/actions/#update-attributes:

This profile establishes the extension link relation https://jsonapi.org/profiles/drupal/actions/#update-attributes (henceforth, only written as update-attributes for readability). A client application processing an update-attributes link MUST process it according to the semantics for updating resource attributes. The client MUST send a request document containing a resource object with a type
and id value equal to the the resource object of the link's context as its primary data. Additionally, the resource object MUST contain exactly the same object as is given by the link's attributes parameter as the resource object's attributes object. Finally, the client MUST not attempt to update any other values while processing the link.

So, a publish link would be represented like so:

{
  "type": "node--article",
  "id": 42,
  "links": {
    "publish": {
      "href": "https://example.com/jsonapi/node/article/42",
      "rel": [
        "https://jsonapi.org/profiles/drupal/actions/#update-attributes",
        "drupal://jsonapi/extensions/core/links/relation-types/publish/"
      ],
      "params": {
        "attributes": {
          "status": 1,
        }
      }
    }
  }
}

If the client processed this link as I defined it above, it would know it must be a PATCH because that's what the spec defines for updating attributes. The context object is node--article:42, so it would know that's the data it should send. Finally, it would jsut directly set the attributes member with the provided object. You'd get:

PATCH /jsonapi/node/article/42

{
  "type": "node--article",
  "id": 42,
  "attributes": {
    "status": 1,
  }
}

Because of the way JSON:API only updates the fields that are provided in a PATCH request and leaves others unchanged, this is super powerful and simple for a client to implement.

should we model them after solid patterns like [UBER]

Honestly, I haven't looked at that spec, so I can't answer completely.

In general, I think I want stay as faithful to RFC standards as possible. Using link relations and the web linking spec for as much as they'll give us. IMO, simplicity is the name of the game. I want to avoid creating (or using) anything that tries to do all the things, in favor of something that is leap forward in capability but also incrementally adoptable and familiar. IOW, it's better to have something with less features if it means more people will use it.

I realize that might read as me being super negative about UBER, but it's not meant to be! Like I said, I have not even read it. It might be exactly what I'm looking for; I'm just laying out a framework for evaluating any possible solutions :)

@Wim Leers asked me to summarize where this issue stands...

Since I wrote the issue summary, we've made lots of progress. Less than I would have hoped in the same time frame, but significant.

What happened related to hypermedia in the last six months (roughly, in no particular order)?

1. #2997277: Place all URLs under the `href` key prepared us for a richer hypermedia feature-set w/o breaking existing clients.
2. #2995960: Add a Link and LinkCollection class to support RFC8288 web linking. added an API-ready object for handling rich links and coupled links to the top-level object.
3. #3033473: Clean-up: Remove LinkManager removed the deprecated API for links.
4. #3032468: Compatibility with upcoming JSON:API 2.2 release: JSON:API Extras 3.4 piloted LinkCollections from a DX perspective.
5. #3015438: Wrap entity objects in a ResourceObject which carries a ResourceType coupled links to resource objects.
6. #2992836: Provide links to resource versions (entity revisions) showed how Links and LinkCollections can be used to provide version states. HATEOAS FTW!
7. #2853066: Spec Compliance: Inaccessible collection/related resources surface errors: should be 200 with hypermedia + metadata converted meta.errors to be hypermedia focused.

What's left/impacted?

1. #3036285: Add a \JsonApiResource\Relationship object to carry relationship data, metadata and a link collection. links need to be coupled to a concept of relationship objects. The top level and resource objects are already taken care of. I think that means we'll probably need a JsonApiResource\Relationship object. I envisioned the DX for this idea in this comment.
2. #3036279: Clean-up: Use a LinkCollection for document omission links. we should convert omissions to a LinkCollection so we're not maintaining complicated deduplication logic in two places.
3. #3032787: [META] Start creating the public PHP API of the JSON:API module: I think hypermedia is a great candidate for this issue. Links and LinkCollections are a nice DX for the API, but I think the mechanism by which we alter LinkCollections today (by decorating normalizers) will not scale. I think we need a plugin system.
4. #2794431: [META] Formalize translations support: the "proper" way to provide translations is via an Accept-Language negotiation while using the hreflang target attribute of resource objects' self links. This would be very similar to #2992836: Provide links to resource versions (entity revisions).
5. #2984628: Consider how to communicate allowed methods on resource object self links is in limbo.

What are the obstacles I see?

1. github.com/json-api/json-api: Introduce RFC8288 Web Linking to v1.2: will make rich linking possible within the base spec. Otherwise, we need to author a profile.
2. We need to standardize on a URI scheme. There's already some context about that in this issue. URIs came up again here: #2857730-48: Add a link to get only published entities in collections.

Update: Added some missing issue links.

I added two new issues and updated the list of issues in #27.

Updated #2984628-11: Consider how to communicate allowed methods on resource object self links.

Closed #3014704-13: Expose API to allow links handling for entities from other modules

@gabesullice, I believe that with the existence of https://www.drupal.org/project/jsonapi_hypermedia and even stable releases, this can now be considered "done"?

I think this issue has served its purpose and marking it "Fixed". Although I don't think the hypermedia story is completely fleshed out yet, this issue is no longer needed to plan/sequentialize/track fledgling hypermedia issues.

In the long term, JSON:API Hypermedia should be merged into the Drupal core JSON:API module to to finalize "the hypermedia story". I think that avenue is open to us. The PHP API is being expanded and the APIs pioneered by JSON:API Hypermedia are being validated by various modules in the ecosystem (see JSON:API Comment for one example).