Allow REST resource config entities to specify a default format

Technically, #2821701: Default '_format' for REST resources routes predates this by a long shot. But it was in the wrong issue queue. So closed that in favor of this.

Its IS said:

I created a RestResource plugin that exposes some data in JSON format. The problem I'm having is that when I access the defined URL for that plugin, I also have to specify the "_format=json" parameter.

Is there a way to alter the default '_format' so that it's json, instead of html, so I can ommit passing it in the URL?

And @dawehner remarked this was kind of similar to #2831137: Remove the need for ?_format=api_json: assume the use of the 'api_json' format for routes managed by JSON API for the JSON API module.

@maosmurf: thanks for reporting this issue, and for the patch!

While I understand this makes things easier, this also comes with some consequences:

1. when the default format configuration is changed, then all user agents (browsers, but also intermediaries such as proxies and reverse proxies) will have cached the previous format
2. this violates RESTful principles
3. this especially violates RESTful principles because the REST module is specifically designed to expose resources in multiple formats
4. this also won't help for many resources that already have a HTML route: GET /node/1 will still return the HTML representation, not the JSON representation. Which means this would only work for some REST resources. Which is very confusing.

I'd like to do say "YES!", but the downsides above almost force me to politely decline…

For the JSON API module this is different, because it's specfically exposing a different (path-prefixed) URL structure and is designed to only ever support one format. So that module can get away with it, but for the REST module this is not a great fit.

So, I think what would make most sense is to create a contributed module that gives you this ability. Storing the default format per resource is exactly what the third_party_settings key in Config Entities' config schema exists for, so it's perfectly possible!

That's my vote: make this a third-party module.

What do others think?

Also, Crell and other "RESTful/HTTP purists" would strongly condemn the use of the term "endpoints" here :P

@maosmurf

I think this is a great idea, however I have to agree with @Wim Leers on this. The inconsistencies this would introduce would be a bit too painful. I think the suggestion of creating a contrib module to support this behavior though is incredibly valuable.

True story, Wim. :-)

I would largely agree with Wim in #4. Were we using HTTP headers as we originally intended (before we realized just how badly broken browsers and caches were), the format would be required (as a header). In case it was missing the routing system itself, I believe, would assume text/html, because that is in practice what most of Drupal's responses are. Changing the default, or making the default sometimes one thing, sometimes another, but as an external user you can't tell which is which introduces a BC change and/or a much uglier and less predicable DX.

-1

Agreed with Wim. Having a default format does only make sense for routes that don't serve HTML. Otherwise the default format will be HTML (for example /node/1).

Instead, I would propose that if there is no HTML route on the same path then REST module should serve just the first format that is defined per default. That way you get your JSON if you omit the _format in the URL. That would also avoid useless 406 Not Acceptable responses when there are only REST routes on the path anyway. Just serve anything instead of an error, right? This also does not violate the HTTP spec where it says that you SHOULD return a 406 when you can't comply to the specified format. A SHOULD is not a MUST, so I think we can just throw JSON in your face if you can't specify formats :)

I think that, if anything, we should allow to drop _format if there is only one format supported for a given route and it's not HTML. So +1 to allowing a default format per route, although maybe only in some contexts.

The real challenge is how to use that format for errors that happen before the route has been matched to fetch the default format. I'm thinking about a route that sets its default to json but an error happens before we can know that about the route. In that case the error would not be json, but html. This error formatting business feels like an edge case and it should not preclude us from having a very useful feature.

Instead, I would propose that if there is no HTML route on the same path then REST module should serve just the first format that is defined per default. That way you get your JSON if you omit the _format in the URL.

But this means that if the configuration changes, or even if just the order of formats in the configuration changes, the default changes. In other words: the concern I raised in #4.1:

1. when the default format configuration is changed, then all user agents (browsers, but also intermediaries such as proxies and reverse proxies) will have cached the previous format

So IMHO we can't do that either. Thoughts?

The real challenge is how to use that format for errors that happen before the route has been matched to fetch the default format. I

Failing so early means we can't really do anything about it. I'm not concerned about this aspect at all: that would not block this from happening! :)

The trick with #8 is that we then need to determine if there is also an HTML route, in which case we could default to the first entry/only entry. I don't know off hand how easy that is.

Also, REST module may not be the only one that's adding a route at a given path. It may only be adding one JSON route, but there's nothing preventing an entirely different module adding its own, say, SVG route at the same path, which REST module would not reliably be able to detect. That's true even if we're not talking about Entities, which we know add an HTML route.

In practice I don't know that a default is going to be possible.

If we want to support the general possiblity to do that, we should expose that somehow in the routing system. We already know how fragile it is, so suggesting contrib to figure out the problem seems kind of ignorant.

I totally think we should though not automagically adapt the behaviour of existing routes. Explicit URLs, either via a prefix (like jsonapi) or via _format IMHO is for itself a good thing.

Failing so early means we can't really do anything about it. I'm not concerned about this aspect at all: that would not block this from happening! :)

As far as I understand @e0ipso cares about 401/403/404 errors, as they ideally would be rendered in a jsonapi format as well?

Did we ever considered again to explore the idea of path prefix for our rest APIs, like /api. Seems wrong to start the discussion again.

As there is no default currently, I don't see an issue regarding backward-compatibility. If defautl is changed later on, one obviously would have to deal with consequences.

The current behavior when you omit ?_format is to always return a 4xx response. This would change that to a 200 response.

None forces you to set default_format on your routes.

Now, you're talking about routes. But this is filed against the REST module. And the IS also explicitly mentions the REST module's configuration. This would be okay to add as a feature to the routing system. But adding it to the REST module is what would be problematic.

Absolutely true, that's why adding functionallity of default_format does not change anything, as long as you do not add default_format explicitely to the routes you'd like to have a default.

Hm, so it sounds like you are proposing to optionally allow REST resource config entities to specify default_format, but to not automatically set it by default. So for example, we would automatically update the node REST resource config entity from:

langcode: en
status: true
dependencies:
  module:
    - basic_auth
    - hal
    - node
id: entity.node
plugin_id: 'entity:node'
granularity: resource
configuration:
  methods:
    - GET
    - POST
    - PATCH
    - DELETE
  formats:
    - hal_json
  authentication:
    - basic_auth

to:

langcode: en
status: true
dependencies:
  module:
    - basic_auth
    - hal
    - node
id: entity.node
plugin_id: 'entity:node'
granularity: resource
configuration:
  methods:
    - GET
    - POST
    - PATCH
    - DELETE
  default_format: false
  formats:
    - hal_json
  authentication:
    - basic_auth

But then a particular site is able to mark a particular format as the default.

This way, BC is kept, we just add a new capability.

Is this a correct representation of what you would like to see?

reroll of #2 for 8.3.0

reroll of #15 for 8.5.x

@Wim: yes thats how the format currently looks like with the patch.

Sorry for the silence here! 😭

The good news is that the reason for that silence is that we've been working hard on lots of foundational issues. Quite a few still remain, but a lot of them have been solved. Including one that was pretty much a blocker: much, much more test coverage, to ensure we don't introduce accidental BC breaks!

In #4 — #12, we were tending towards not doing this, for the reasons outlined there.

But then @maosmurf posted #13, I asked for clarifications in #14, and @kfritsche confirmed. I think this general approach may be okay. However … since then, #2854560: \Drupal\Core\Routing\RequestFormatRouteFilter::filter() is too HTML-centric landed. And that already introduced something fairly similar. If there's a single route match, and that route match uses a format other than html (say foo), then we match that route using the foo format, even without specifying ?_format=foo in the URL.
It won't be effective without #2854543: NegotiationMiddleware calls $request->setRequestFormat('html') when there is no _format request parameter, but shouldn't. But if you apply #2854543-7: NegotiationMiddleware calls $request->setRequestFormat('html') when there is no _format request parameter, but shouldn't to HEAD, and configure a REST resource with a single format (for example the entity:node_type one), then it does what you want: http://d8/entity/node_type/article returns a JSON response, without a ?_format query string!

Doesn't that give you what you're asking here? That'd allow us to not add more things to configure in REST resources, which would keep it more maintainable :) If it does, please help out with #2854543: NegotiationMiddleware calls $request->setRequestFormat('html') when there is no _format request parameter, but shouldn't! If it doesn't, then please clarify why it doesn't fulfill your needs.

P.S.: @maosmurf I overlooked First and most, thank you for your feedback (especially, as this is my first issue on drupal.org) — congrats, and thank you! 👍

#2854543: NegotiationMiddleware calls $request->setRequestFormat('html') when there is no _format request parameter, but shouldn't landed yesterday!

I think that already gives us what we want! See #20 for detailed rationale. In other words: the feature request made sense at the time, but thanks to #2854543, it's no longer necessary.

In 8 days, it'll have been six months without a single reply to #20, and nearly eight months since anybody besides me commented on this. That further indicates that this likely is no longer necessary. Therefore marking this Closed (outdated), since it's no longer relevant.

Thanks to everyone who contributed to this issue!

I know this is a very old issue that’s been closed for almost three years, but I believe it is still a problem.

If you follow the instructions exactly as written on Custom REST Resources, it works as expected.

However, if you both:

1. Allow more than one format, e.g., application/json and application/xml
2. Do not add the _format query parameter to the request

The response is HTTP 406 Not Acceptable.

Doing both of those things at the same time is not what I would consider an unlikely scenario - all you would have to do is check two boxes instead of one, and then not append ?_format=json or ?_format=xml to the URL.

tl;dr - I believe this feature request should be reopened on the grounds that it is very easy to produce an unexpected result (HTTP 406).

Agreed, a default format really would solve this problem. I like the patch in #17 for this - adjust and look at getting it into 9?

Here's a patch that uses the accept headers as decriebed in #25

Could be worked into the main issue of having a default format or on its own to leverage http headers to set the request format instead of that argument.