(use case #2) Decide on an API response format for menu data

I've included my proposed API in the issue summary. Note that the use of title as a sibling of href is using version 1.1-rc3 of the JSON:API spec. However, as a spec editor, I'm confident that this will not change before 1.1 is finalized.

An advantage of using 1.1 link objects is our ability to use target attributes to give hints to the consumer about how the menu link should be treated. For example, let's imagine that a menu contains a link to a non-JSON:API route, such as /admin/content or /node/3/edit, the link can hint to the client that it cannot make a JSON:API request for it by using the type target attribute:

{
  "href": "/admin/content",
  "title": "Edit",
  "type": "text/html",
  "meta": {
    "menu": "shortcut",
    "hierarchy": ".001"
  }
}

Fully-decoupled consumers could choose to filter these links out and progressively decoupled apps could just to follow them by causing an actual browser navigation (e.g. window.location = link.href). This could be very useful in Drupal core for progressively moving to a more decoupled Admin UI, for example. It could also be handy in providing an authenticate link in an OAuth redirection.

We will be using the hreflang target attribute for a similar purpose in #2794431: [META] Formalize translations support.

Thanks for the comparison!

My concern with that serialization is that it's exposing a lot of back-end implementation detail and complexity. Relying on any of: provider, enabled, route names and parameters, or menu item IDs suggests that the front end is tightly coupled to the back end.

For example, if items which aren't enabled are not omitted by the back end, then the front end implementation necessarily becomes coupled the the idea that links can be enabled or disabled which means that the front-end implementation has to be more complicated and larger since it has to maintain the code to remove disabled menu items on its own[1].

expanded is the most notably absent value since parent and weight are implied by the hierarchy value in the proposed format[2].

I intentionally excluded expanded since I felt like it was encoding a presentation detail that ought to be the concern of the front end... but I don't feel strongly about that. I can understand why we would want to include it in order to give content editors more control over the front end presentation... In fact, it wouldn't take much to convince me that it should be included since the Decoupled Menu Initiative is about eliminating compromises when choosing to build a decoupled site. What do you think @Deciphered?

[1] Writing this made me go take a look at JSON:API Menu Item's implementation again. It looks like the back end is omitting disabled menu items, which makes enabled meaningless because it will always be true.

[2] The reason I proposed a hierarchy value instead of using parent and weight is because it makes it trivial to reconstruct the tree quickly and because using parent means you also need to have an id for every item. IOW, having the single hierarchy value means we can eliminate two other values and avoid writing recursive code to reconstruct a tree on the front end.

Thanks for trying to deconstruct the discussion into more manageable parts @e0ipso!

Resource data vs link data

One works towards homogenization and minimizing astonishment by staying consistent with existing resource types …

I think it works the same way for nodes, so we should stick to the same pattern everywhere.

I think a good analogy here is from Menus to Views. If we were to add an endpoint for Views, there are two ways that we might want to do that:

1. Expose the View config entity and all its filters, relations, sorting rules, etc. so that a decoupled front end could reconstruct the configured query and then execute a JSON:API request for the same data.
2. Execute the View and render a JSON:API representation of the result.

If we represent all menu items as resource objects, then I think we're choosing the former approach of exposing the data and expecting the decoupled front end to interpret it and to recreate the end result. This is an RPC-like approach.

If we represent all menu items as link objects, then I think we're choosing approach #2 which interpret the configuration on the back end and exposes the featureful result in a JSON:API format. My argument is ultimately that this approach simplifies consumer implementations and makes it easier to add new features and fix bugs. This is a more RESTful approach.

All that said, the question of whether we should represent menu items as resource objects or link objects presents a somewhat false premise: that it must be one or the other exclusively. In fact, we already expose most of the data because JSON:API exposes menu config entities as well as the menu_link_content entities. The only thing we do not expose is the menu items provided in code and in YAML. This makes me wonder if what is really lacking here is an API for plugin data, which, incidentally, is already available via the JSON-RPC module.

However hypermedia links are transitive, they need an object to refer to. Right now there is no implicit context these links would refer to. It seems there is a missing discussion to be had if we want to nail semantics here.

This is a good point. It is semantically wrong to put links on the menu config entity because it implies the links go from the config entity resource to the targeted resources. Perhaps we need to instead "attach" menu links to response documents using logic similar to the block placement system which presently attaches menus blocks to content pages (actually, this is similar to how the JSON:API Navigation sandbox works today).

This raises an extra argument for links, IMO, if the HTML page for /node/1 represents an article and has a primary menu block on it, both the article and the menu data are available in a single request. However, if we make separate endpoints for /jsonapi/node/article/{id} and menus, then one must instead make two requests: one for the article and one for the menu (or we must construct an artificial JSON:API relationship that can use the ?include parameter)

Hierarchy

There is a gap in how to imply hierarchy: let's define the features we want to support then let's define how each approach will look like. That will help to decide better. For instance: "give me the menu subtree starting on the Link12 menu link item in a depth of 3, with 2 levels of depth".

I would like to support the association of parents with children and subtrees and depth, but I would not like to support expansion. IMO, the first three are functional features and expansion is purely presentational (I don't think JSON:API should be concerned with presentational data).

Framework logic

Dumping a "created_day: Wed" property and hoping for all consumers to follow suit is unacceptable and unrealistic (IMO). Bear in mind that you cannot force client updates in all client side technologies (phones, TVs, billboards, kiosks, in-seat entertainment systems, etc).

I think this is an incredibly important point. We may be building this API to support a JavaScript library we maintain, but the API needs to be as functional as possible for non-JS consumers. We cannot build components for every device, so I think we need to keep as much business logic as possible on the back end so that it doesn't need to recreated for each consumer-type.

@Deciphered, I think we have different use cases/architectures in mind as we think about what to expose and how to expose it. You shared your opinion about how things should be:

My opinion across the board is that any information that can be provided, should be provided. A decoupled application should be able to chose to use the data, just the same way a Drupal module can access and render the data… You should keep back the same sort of business logic that would would keep back from Twig. The frontend component should be able to consume the required config and content to offer a similar experience to that of Twig in Drupal. But that should go for both frontend and backend Twig, as it would be ideal if we could use the output to display a menu as well as manage a menu.

Can you explain why you believe these things should be the way you say they should be? I'm afraid that we will fundamentally disagree that "any information that can be provided, should be provided," for this initiative, at this moment, unless I'm missing some compelling motivations.

Let me try to explain why I think we should not expose everything:

A major motivation for this initiative was to re-empower content editors even in decoupled scenarios. Drupal's status quo means that when you choose to build a decoupled site, your content editors sacrifice many of the controls they're used to in a "coupled" Drupal-rendered site (e.g. the ability to order/disable menu items or use the block system).

If we take the approach that we should expose the config so that the a consumer can read it and interpret it, then we're saying that the consumer has to read it and interpret it. This means that a consumer might not notice the enabled field or postpone implementing it to save time, which would disempower the content editor because that field would no longer have any effect.

As the author of Druxt, this distinction probably seems irrelevant. Druxt handles the enabled field properly, so it's a non-issue. Moreover, when we ship a core JavaScript library for menus, it can contain the framework logic that correctly handles the enabled field. That's entirely logical.

However, since this initiative is no longer called the JS menu component initiative, I think that means that means that our HTTP API needs to consider non-JS use cases. In those cases, Drupal core will not have the luxury of shipping platform-specific libraries to handle its framework logic correctly and Drupal cannot rely on projects like Druxt to interpret the HTTP API as it intended.

In other words, the more framework logic we expect consumers to write, the more logic those implementations will fail to implement or implement incorrectly. And the more framework logic we've forced them to learn (e.g. they'll ask "what does provider even mean? Is it something I need to handle?"). Therefore, I think we need to take a very simple, read-only approach that leaves fewer decisions for consumers to make. I think we want to lower the cognitive overhead and let the consumers maintain less complicated code.

Providing all the information that can be provided and letting the consumer choose how to use that information runs counter to this idea and it will lead to disempowered content editors whenever the consumer chooses not to respect the information it have been provided.

EDIT This reply was written after #13. I haven't yet read the later comments.

Could you please provide any other argument besides "decoupled app developer needs to be safeguarded?".

@mirom, I feel like I must be doing a really terrible job of explaining my position if you can summarize everything I've written as "safeguarding" decoupled app developers". It's like saying that a developer writing a PHP class with protected or private properties is just trying to "safeguard" its users.

@gabe, I think that what you're suggesting diverges from the common practice for other JSON:API endpoints.

The existing JSON:API endpoints do one thing: they serialize entities.

A menu isn't a collection of entities though. A menu is a set of links. My position is that we should render menu links... as JSON:API links.

It's unfortunate that in Drupal 8 and above, some links are created by creating menu_link_content entities and so there is an existing JSON:API endpoint for them because there are endpoints for all entities (I regret this, BTW).*** I think that it's this existing endpoint that is giving you the impression that I'm trying to diverge from common practice.

If you start with the question "how should we expose all menu items?" then it seems reasonable to make "fake" entities for menu items that aren't provided by menu_link_content entities and add them in with the other ones, but that would probably be a bigger change to common practices and would open a huge can of worms.

However, if you start with the question "what are menus and how should we add them to JSON:API?" then we get a pretty straightforward answer: menu links should be added as JSON:API links to JSON:API responses just like menu links are added as HTML links to HTML responses.

*** In case it's not clear, I want to keep those endpoints and I do not want to change them. My proposal is purely additive.

Could we provide both the menu link content and the node under 'relationships' @gabesullice?

This wouldn't work because I'm proposing that we should "render" menus as link objects, which don't have attributes or relationships.

I think this is the key distinction you might be missing @larowlan. The biggest difference is not what properties to show or not show, it's about how JSON:API treats menu links altogether. I.e. should it treat them more like content (and therefore resource objects) or should it treat them more like navigational hypermedia (and therefore link objects)?

This also has implications to how I want to reply to you here:

I.e. if the route uri uses the entity scheme (entity://) it would be good to pass on that information as it might help frontend code decide how to resolve the URL

If we treat them as hyperlinks, then the way we should signal "how to resolve the URL" is through link relation types (which I've added to the JSON:API 1.1 spec itself specifically so that Drupal can use them 😊)

Sure! Let's imagine we wanted to add a shortcut link to a frequently edited node (a landing page, for example) in a way that makes it super simple for a JS rendered web-app to direct the user to the Drupal-rendered edit page. That could look like this:

{
  "href": "/node/3/edit",
  "title": "Edit",
  "type": "text/html",
  "rel": "edit"
  "meta": {
    "menu": "shortcut",
    "hierarchy": ".001"
  }
}

This link tells the consumer "don't try to fetch this! It's HTML! Just link to it with an <a> element". I know, type is a target attribute not a link rel, so, let's say that the consumer has been "taught" that edit links should all use have target="_blank". That's one admittedly contrived example :) it would make more sense for contextual links, but if we do this for menus, it will be an easy pattern to replicate for contextual links.

@nod_, thanks for jumping in! I'm a bit confused though.

Can you explain the distinction? I'm worried about splitting the discussion along lines that aren't clear to me.

No worries, thanks.

One thing isn't clear to me. You're saying that you want to provide a list of links that can be rendered at the frontend, is that correct?

Yeah, pretty much correct :) *

So if I continue with my use case from #17, then how can I render that icon at the frontend?

I went and read the Menu Item Extra module's code and it looks like it's providing a new, fieldable entity type and even replacing the menu form. So, I don't think it's something we would want to directly support in core.**

However, I can think of two ways to handle it in contrib:

1. menu_item_extras could alter the JSON:API link and something to the meta property of the link object (it's already altering the way menu links are rendered in Twig. This is close to what @larowlan is saying in #14 and what you were suggesting when you said Are we going to provide at least some menu link content entity id?
2. JSON:API will continue to support the /jsonapi/menu_link_content/menu_link_content endpoint that serializes those content entities. For this tightly coupled use case, I think the front end could use that directly.

Perhaps a combination of 1 and 2 would work. menu_item_extras could add the URL of the menu content entity to the link's meta.

* I want to provide a list of menu links. In most cases, they will be rendered on a front end, but they should also be able to navigate non-rendered things, like a command line client or a phone menu.

** I'd also argue that adding icons via menu items on the backend is beginning to break the front end/back end idea about decoupled architectures, which usually separate presentation (e.g. icons) from content and functionality. Presentation being a front end concern and the rest a back end concern.

@pdureau, for better or for worse, I think render arrays are too coupled to presentation logic and they also have weird functional things, like the #access property and #pre_render callbacks, which we wouldn't want to send over HTTP. I like to think that we "render" JSON:API responses the same way that we render HTML responses.

I agree with you about keeping site-building abilities. My proposal about rendering menu links tries its best to do that.

Today, the pipeline is roughly like this:

Menu item entities
                  \
        Menu link plugins -> Menu tree tree elements -> Render array -> HTML
                  /
YAML-defined links

and my proposal would look something like this:

Menu item entities
                  \
        Menu link plugins -> Menu link tree elements -> JSON:API
                  /
YAML-defined links

I think that as long as we preserve the enabled/disabled information, hierarchy, order, and replicate the visibility rules that menu blocks have for JSON:API, then we really will have all the same back-office capabilities as before, even in a decoupled context since putting together the menu link tree is where it all that config gets brought together (except for the visibility conditions).

As @nod_ suggested in #23, we created a meta issue to "frame" the two different ways one might want to consume menu data. You can read the summary here: #3196329: [META] Use cases covered by the decoupled menus initiative

In short, we agreed that we need two approaches on the back end to cater to two different ways of consuming menu data.

1. For those for whom their menus are global and static (i.e. appear independent of context and do not change from page to page), I made a proposal in #3189459-4: (use case #1) Provide an API response for "stateless menu" use case and we can discuss it there.
2. For those who have complex and contextual needs (i.e. only show this menus for users with a particular role) and want site builders to have more control over menu placement, we can use this issue. I think the issue summary needs to be updated on this one so that it's more tightly scoped. The proposal is also not ideal for the use case in mind since the IS proposal suggests additional endpoints.

I've updated the issue summary to better reflect the problem that this issue is trying to solve. I also added a more fleshed out proposed resolution.

Some things to note:

1. The links are on the "top-level" object, not the node object.
2. Hierarchy is expressed via the children meta property, this is so that it has at least some symmetry with the proposal in #3189459-4: (use case #1) Provide an API response for "stateless menu" use case. The array values are equivalent to the links object key (e.g. drupal:menu:item:1)
3. The href is a URL alias, not a JSON:API-specific URL. This is because #3196329: [META] Use cases covered by the decoupled menus initiative proposes to solve URL aliases by using Accept header negotiation (see that issue for an issue link).
4. The link key drupal:menu:item:2 is not ideal. Ideally, we would use a valid link relation type.
5. The title key is only "available" in JSON:API 1.1 (Drupal has only implemented 1.0 so far), but since yours truly gets to help decide what is considered valid JSON:API format, I promise this is fine ;)
6. You will have to decide for yourself the best book of all time is.
7. This format is verbose. But it's no more verbose than any other HTML would be. In fact, this is a drawback of all RESTful systems that we've known about since the very beginning...
   • The trade-off, though, is that a uniform interface degrades efficiency, since information is transferred in a standardized form rather than one which is specific to an application's needs. The REST interface is designed to be efficient for large-grain hypermedia data transfer, optimizing for the common case of the Web, but resulting in an interface that is not optimal for other forms of architectural interaction. — Roy Fielding
   • A future performance optimization could be to allow site builders to opt into document links as separate, linked "linkset" resources. This feature could follow this IETF draft: Linkset: Media Types and a Link Relation Type for Link Sets
8. This proposal can also serve global/static menu use case too; there's no inherent reason it wouldn't work. It's simply more verbose to send so much data on every request if it's not strictly needed and may not be the intuitive approach for developers who are not bought into the "Drupal way" (if this becomes one of Drupal's "Ways" of course 😉).