Problem/Motivation

In Change Notice https://www.drupal.org/node/2199185 based on #2019123: Use the same canonical URI paths as for HTML routes we missed the change to POST request.

Some route like rest.entity.node.POST have paths prepended by /entity like /entity/node which is not in line with the other methods GET, PATCH, DELETE.

Furthermore, for entity types that have multiple words in their name, it's often even worse: GET /taxonomy/term/42, PATCH /taxonomy/term/42 or DELETE /taxonomy/term/42, but POST /entity/taxonomy_term (note the prefix and the underscore).

Proposed resolution

  1. POSTing entities via REST should happen at /{entity_type} (e.g. /node, /comment …), not at /entity/{entity_type} (e.g. /entity/node, /entity/comment …).
  2. But we shouldn't hard code this, we should use the same opt-in mechanism that we already use for canonical: entity types that specify a canonical link template will be available at that same path via REST. We should do the same for <code>https://www.drupal.org/link-relations/create.
  3. Provide BC through a path processor.

In other words: just like the REST module in Drupal 8.0.x, 8.1.x and 8.2.x has already been using the canonical link template of an entity type if it is specified in favor of its default (/entity/{entity_type}/{entity}), starting in Drupal 8.3.x, it will do exactly the same for the create link template (for the https://www.drupal.org/link-relations/create link relation type): it will use that instead of the default (/entity/{entity_type}).

Remaining tasks

  1. Blocked on #2730497: REST Views override existing REST routes
  2. Fully functional patch.
  3. Small test to prove that the BC layer works.

User interface changes

None.

API changes

None.

Data model changes

None.

CommentFileSizeAuthor
#239 2293697-239.patch24.07 KBWim Leers
#233 2293697-233.patch24.04 KBWim Leers
#230 2293697-230.patch24.09 KBdawehner
#223 2293697-223.patch23.9 KBjofitz
#223 interdiff-221-223.txt864 bytesjofitz
#221 2293697-221.patch23.86 KBjofitz
#211 interdiff208_211.txt1.9 KBMunavijayalakshmi
#211 2293697-211.patch24.35 KBMunavijayalakshmi
#208 2293697-208.patch24.35 KBjofitz
#206 2293697-206.patch25.62 KBWim Leers
#206 interdiff.txt3.28 KBWim Leers
#194 interdiff-2293697-194.patch2.13 KBdawehner
#194 2293697-194.patch22.23 KBdawehner
#188 2293697-186.patch21.25 KBWim Leers
#188 interdiff.txt6.21 KBWim Leers
#177 2293697-177.patch23.91 KBWim Leers
#177 interdiff.txt4.03 KBWim Leers
#172 2293697-172.patch27.46 KBWim Leers
#172 interdiff.txt1.42 KBWim Leers
#171 2293697-171.patch27.36 KBWim Leers
#171 interdiff.txt1007 bytesWim Leers
#168 2293697-168.patch27.35 KBWim Leers
#168 interdiff.txt1.15 KBWim Leers
#167 2293697-167.patch26.24 KBWim Leers
#165 2293697-165.patch30.29 KBWim Leers
#165 interdiff-152-165.txt3.86 KBWim Leers
#157 2293697-157.patch30.51 KBWim Leers
#157 interdiff.txt4.28 KBWim Leers
#152 2293697-152.patch26.49 KBWim Leers
#152 interdiff.txt1000 bytesWim Leers
#151 2293697-150.patch26.41 KBWim Leers
#151 interdiff.txt3.46 KBWim Leers
#148 2293697-148.patch23.48 KBWim Leers
#148 interdiff.txt593 bytesWim Leers
#124 2293697-124.patch10.32 KBWim Leers
#124 interdiff.txt8.35 KBWim Leers
#119 interdiff-removed-test-coverage.txt11.21 KBWim Leers
#119 2293697-119.patch7.03 KBWim Leers
#105 2293697-105.patch18.38 KBWim Leers
#105 interdiff.txt7.18 KBWim Leers
#84 2293697-73.patch15.15 KBdawehner
#81 interdiff.txt738 bytesdawehner
#81 2293697-81.patch14.24 KBdawehner
#79 interdiff.txt1.29 KBdawehner
#79 2293697-79.patch14.96 KBdawehner
#73 interdiff.txt738 bytesdawehner
#73 2293697-73.patch15.15 KBdawehner
#68 interdiff.txt5.15 KBdawehner
#68 2293697-15.patch14.43 KBdawehner
#50 interdiff.txt5.92 KBWim Leers
#50 rest-post-canonical-2293697-50.patch10.61 KBWim Leers
#48 interdiff.txt1.99 KBdawehner
#46 2293697-46.patch9.58 KBdawehner
#37 interdiff.txt1.35 KBWim Leers
#37 rest-post-canonical-2293697-36.patch9.31 KBWim Leers
#35 rest-post-canonical-2293697-35.patch8.05 KBWim Leers
#23 rest-post-canonical-2293697-23.patch7.7 KBhchonov
#20 rest-post-canonical-2293697-20.patch5.35 KBvedpareek
#17 rest-post-canonical-2293697-17.patch4.67 KBvedpareek
#16 rest-post-canonical-2293697-16.patch4.67 KBvedpareek
#12 some_test_entities-2293697-12.patch7.14 KBclemens.tolboom
#7 some_test_entities-2293697-7.patch603 bytesclemens.tolboom
rest-post-canonical.patch5.18 KBclemens.tolboom
#126 interdiff.txt3.4 KBWim Leers
#126 2293697-126.patch13.64 KBWim Leers
#127 interdiff.txt7.18 KBWim Leers
#127 2293697-127.patch20.28 KBWim Leers
#128 interdiff.txt1.65 KBWim Leers
#128 2293697-128.patch20.25 KBWim Leers
#129 interdiff.txt1.39 KBWim Leers
#129 2293697-129.patch21.15 KBWim Leers
#130 interdiff.txt6.31 KBWim Leers
#130 2293697-130.patch21.44 KBWim Leers
#138 2293697-131.patch20.38 KBtedbow
#138 interdiff-130-131.txt1.49 KBtedbow
#146 interdiff.txt1.5 KBWim Leers
#146 2293697-146.patch22.92 KBWim Leers
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

clemens.tolboom’s picture

Issue summary: View changes

Added SQL + result

clemens.tolboom’s picture

Title: Rest POST not following Change Notice » rest.node.POST not following Change Notice
Issue summary: View changes

Added rest.node.POST

chx’s picture

Title: rest.node.POST not following Change Notice » Some test/entities still REST at entity/$entity_type
Status: Needs review » Needs work

Here is some supporting evidence:

core/modules/rest/src/Tests/NodeTest.php
65:    $this->httpRequest('node/' . $node->id(), 'GET', NULL, 'application/json');
87:    $this->httpRequest('node/' . $node->id(), 'PATCH', $serialized, $this->defaultMimeType);
core/modules/rest/src/Tests/UpdateTest.php
140:    $this->httpRequest($entity_type . '/9999', 'PATCH', $serialized, $this->defaultMimeType);
core/modules/rest/src/Tests/DeleteTest.php
62:      $response = $this->httpRequest($entity_type . '/9999', 'DELETE');

Indeed it looks this is correct. However the patch missed two:

core/modules/rest/src/Tests/DeleteTest.php
79:    $this->httpRequest('entity/user/' . $account->id(), 'DELETE');
core/modules/rest/src/Tests/CreateTest.php
123:    $this->httpRequest('entity/entity_test', 'POST', $serialized, $this->defaultMimeType);
clemens.tolboom’s picture

Assigned: Unassigned » clemens.tolboom

@chx thanks

Question now is what endpoints should not start with /entity?

In #2292707-10: GET on entity/taxonomy_vocabulary/{id} is not working I enabled all possible endpoint then checked for the non /entity endpoints.

select distinct path from router where name like 'rest.%' and path not like '/entity%' ORDER BY path;

/admin/config/user-interface/shortcut/link/{shortcut}
/block/{block_content}
/comment/{comment}
/dblog/{id}
/node/{node}
/rest/session/token
/taxonomy/term/{taxonomy_term}
/user/{user}
clemens.tolboom’s picture

Running on my `enable all` install

select name, path from router where name like 'rest.%' ORDER BY path;

gives 178 result (divide by ~5 GET (json, xml), PATCH, POST, DELETE) ~40 'entities'.

chx’s picture

I do not know the generic answer to that but at least user DELETE seems wrong since #4 found a rest route for user/{user} . I would say let's fix that one at least. I absolutely have no idea how or where rest routing is generated as I generally try to stay away from routing. (And in fact, I am out of this issue too.)

clemens.tolboom’s picture

Digging around I found #2019123: Use the same canonical URI paths as for HTML routes which suggests Rest ResourceBase is in err.

So a completely different patch without test fixes yet.

SELECT * FROM router WHERE name LIKE "rest.entity.%"; gives a consistent pattern for node, user, comment and file. File is still on /entity/file which makes me puzzled again as I consider that a 'normal' exposed entity.

Status: Needs review » Needs work

The last submitted patch, 7: some_test_entities-2293697-7.patch, failed testing.

Berdir’s picture

file has now routes in core, so it falls back to the default, that makes sense.

Only question is if the default should not include the leading entity/ now. file is an interesting example. because contrib adds file routes/links, so that would change the REST routes...

clemens.tolboom’s picture

Assigned: clemens.tolboom » Unassigned

I don't get the failures of #7

R.Muilwijk’s picture

#7 Fails because it only patches the setPattern and did not run your patch in https://www.drupal.org/files/issues/rest-post-canonical.patch. Provide a single patch?

clemens.tolboom’s picture

Title: Some test/entities still REST at entity/$entity_type » POST must use canonical endpoint (not prepended with entity/)
FileSize
7.14 KB

Patch needed a combine (thanks @R.Muilwijk), reroll and additions from #3.

clemens.tolboom’s picture

Issue summary: View changes
Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 12: some_test_entities-2293697-12.patch, failed testing.

clemens.tolboom’s picture

Issue tags: +Needs reroll

Rechecking this patch I ran into:

#2406439: Cleanup EntityDerivative and RouteBuilderInterface
#2281645: Make entity annotations use link templates instead of route names

which targets a higher goal.

Checking the current db routes on node, user, comment, file I find the odds

GET: /entity/file/{file}

PATCH: /entity/file/{file}

POST: ALL on /entity/{entity_type} which is better. This is caused by the default

# core/modules/rest/src/Plugin/Deriver/EntityDeriver.php:85
          'http://drupal.org/link-relations/create' => "/entity/$entity_type_id",

DELETE: /entity/file/{file}

I think we can just reroll this patch and check for the odd /entity/file in the issue queue.

vedpareek’s picture

Rerolled

vedpareek’s picture

Status: Needs work » Needs review
FileSize
4.67 KB

Rerolled

dawehner’s picture

Issue tags: +API change

Tagging with API change, as the URLs are now some form of API.

Status: Needs review » Needs work

The last submitted patch, 17: rest-post-canonical-2293697-17.patch, failed testing.

vedpareek’s picture

Status: Needs work » Needs review
FileSize
5.35 KB

Rerolled

Status: Needs review » Needs work

The last submitted patch, 20: rest-post-canonical-2293697-20.patch, failed testing.

dcam’s picture

Issue tags: -Needs reroll

#20 applies to HEAD. Removing the "Needs reroll" tag.

hchonov’s picture

Status: Needs work » Needs review
FileSize
7.7 KB

This one should pass now.

clemens.tolboom’s picture

+++ b/core/modules/rest/src/Plugin/ResourceBase.php
@@ -111,7 +111,6 @@ public function routes() {
-          $route->setPattern($create_path);

I do miss this from latest patch(es). iirc this is now unnecessary. See patch from #12

hchonov’s picture

@clemens.tolboom:
If we remove it, then calling
Url::fromRoute('rest.entity.' . $this->testEntityType . '.POST')->setAbsolute()->toString()
throws an exception :
exception 'Symfony\Component\Routing\Exception\MissingMandatoryParametersException' with message 'Some mandatory parameters are missing ("entity_test") to generate a URL for route "rest.entity.entity_test.POST".' in Drupal\Core\Routing\UrlGenerator-&gt;doGenerate() (line 174

clemens.tolboom’s picture

@hchonov ic ... thanks for testing :-)

Did you do some manual testing too. My angular app is broken :(

hchonov’s picture

@clemens.tolboom
yes, I did manual testing as well. So I guess the patch looks good now?

dawehner’s picture

In case we do that, we should hae some form of change record, given that this is kinda some API change isn't it? I think the REST interface is considered as real API

klausi’s picture

Assigned: Unassigned » Crell
Status: Needs review » Needs work
Issue tags: +Needs issue summary update

This would be a pretty heavy API change at this point. We left out the creation POST route on purpose, because it does not align with any other route that serves HTML (the node creation form for example is on a completely different path).

So we need a pretty strong reason in the issue summary why we should change this at this point, can anyone do that? For backwards compatibility we could register 2 routes that do the same thing, one on /node and the legacy one on /entity/node.

I would close this as won't fix, let's ask Crell, the second rest.module maintainer what he thinks.

Wim Leers’s picture

Issue tags: +API-First Initiative
dawehner’s picture

I would close this as won't fix, let's ask Crell, the second rest.module maintainer what he thinks.

Now a couple of months later there is even less reasons for that. I guess we just have to say no and maybe think about it in D9? Providing some alias as well seems a bit weird.

Wim Leers’s picture

Issue tags: +DrupalWTF

That's almost exactly what I first wanted to post. :)

Then I deleted what I wrote. Because this *is* very weird, a Drupal WTF. It's very counterintuitive. That's why I think an alias would actually be a good thing.

dawehner’s picture

Agreed.

dawehner’s picture

I would even argue at that time the sane option should be the default, and the BC layer should be the alias.

Wim Leers’s picture

Status: Needs work » Needs review
FileSize
8.05 KB

Rebased #23, it no longer applied.

Status: Needs review » Needs work

The last submitted patch, 35: rest-post-canonical-2293697-35.patch, failed testing.

Wim Leers’s picture

Status: Needs work » Needs review
FileSize
9.31 KB
1.35 KB

A regular URL alias won't work AFAIK. I think we can solve this using an inbound path processor? Something like this? Ideally, we'd even only conditionally enable this, i.e. only for sites that were originally installed using 8.0.x.

Status: Needs review » Needs work

The last submitted patch, 37: rest-post-canonical-2293697-36.patch, failed testing.

dawehner’s picture

A regular URL alias won't work AFAIK. I think we can solve this using an inbound path processor? Something like this? Ideally, we'd even only conditionally enable this, i.e. only for sites that were originally installed using 8.0.x.

Yeah, we could have a container parameter to turn off this behaviour. This sounds like a great idea

Wim Leers’s picture

Version: 8.0.x-dev » 8.1.x-dev
Issue tags: -API change

Patch is against 8.1. This should happen in 8.1 anyway, otherwise it's a BC break.

Wim Leers’s picture

Issue tags: +Needs change record

Re-tested, but no idea if it'll now pick 8.1. May need to reupload.

Also, will need a CR.

The last submitted patch, 37: rest-post-canonical-2293697-36.patch, failed testing.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.0-beta1 was released on March 2, 2016, which means new developments and disruptive changes should now be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Wim Leers’s picture

Title: POST must use canonical endpoint (not prepended with entity/) » EntityResource: POST must use canonical endpoint, just like other HTTP methods (not prepended with /entity/)
Assigned: Crell » Unassigned
Priority: Normal » Major

Yet another person who ran into this problem and found it very confusing: #2667186: Can't make a post with cookie authentication.

Wim Leers’s picture

Title: EntityResource: POST must use canonical endpoint, just like other HTTP methods (not prepended with /entity/) » EntityResource: GET/PATCH/DELETE to /node, but POST to /entity/node

Clearer title.

dawehner’s picture

Status: Needs work » Needs review
FileSize
9.58 KB

Let's see whether this works.

Wim Leers’s picture

Nice! This looks great :)

  1. I'm not seeing a service definition for this new class, so I don't think this can work? You probably have that locally, but forgot to include it in the patch?
  2. +++ b/core/modules/rest/src/PathProcessor/PathProcessorEntityResourceBC.php
    @@ -0,0 +1,50 @@
    +    // @todo What happens on language prefixes?
    

    Yeah, I was thinking the same. We definitely need test coverage for that.

    Path processors run in a certain order, so we need to make sure that this runs after the language one.

  3. +++ b/core/modules/rest/src/Tests/CreateTest.php
    @@ -93,7 +93,7 @@ public function testCreateWithoutPermission() {
    +    $this->httpRequest($entity_type, 'POST', $serialized, $this->defaultMimeType);
    
    @@ -138,7 +138,7 @@ public function testCreateEntityTest() {
    +      $this->httpRequest($entity_type, 'POST', $serialized, $this->defaultMimeType);
    
    @@ -187,7 +187,7 @@ public function testCreateNode() {
    +        $this->httpRequest($entity_type, 'POST', $serialized, $this->defaultMimeType);
    
    @@ -406,7 +406,7 @@ public function assertReadEntityIdFromHeaderAndDb($entity_type, EntityInterface
    +    $this->httpRequest($entity_type, 'POST', 'kaboom!', $this->defaultMimeType);
    
    @@ -417,7 +417,7 @@ public function assertCreateEntityInvalidData($entity_type) {
    +    $this->httpRequest($entity_type, 'POST', NULL, $this->defaultMimeType);
    
    @@ -436,7 +436,7 @@ public function assertCreateEntityInvalidSerialized(EntityInterface $entity, $en
    +    $response = $this->httpRequest($entity_type, 'POST', $invalid_serialized, $this->defaultMimeType);
    
    @@ -456,7 +456,7 @@ public function assertCreateEntityInvalidSerialized(EntityInterface $entity, $en
    +    $this->httpRequest($entity_type, 'POST', $serialized, $this->defaultMimeType);
    

    We have a leading slash in the other places, so let's do that here too?

dawehner’s picture

FileSize
1.99 KB

This is the interdiff

Wim Leers’s picture

Assigned: Unassigned » Wim Leers
Status: Needs review » Needs work

LOL, apparently I rolled this patch previously. I'll fix my own remarks then :P

Wim Leers’s picture

Yeah, I was thinking the same. We definitely need test coverage for that.

Path processors run in a certain order, so we need to make sure that this runs after the language one.

Actually, we don't need to do anything. Precisely since the language prefix needs to processed before everything else, the priority of the language path processor is very high. So it's done first. Precisely to allow contrib modules to not have to think about this. So, we can keep the default priority, by not specifying any priority, which gets us priority 0.

Fixed all my feedback in #47.

dawehner’s picture

Looks pretty sane now.

One issue I found is that both \Drupal\views\Plugin\views\display\PathPluginBase::alterRoutes and \Drupal\page_manager\Routing\PageManagerRoutes::alterRoutes gives a damn about HTTP methods, so they potentially override the POST route now.
I think this is simply never the case at the moment, because the default canonical routes are registered first.

The last submitted patch, 46: 2293697-46.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 50: rest-post-canonical-2293697-50.patch, failed testing.

dawehner’s picture

Wim Leers’s picture

Title: EntityResource: GET/PATCH/DELETE to /node, but POST to /entity/node » [PP-1] EntityResource: GET/PATCH/DELETE to /node, but POST to /entity/node
Status: Needs work » Postponed
Related issues: +#2449143: REST views specify HTML as a possible request format, so if there is a "regular" HTML view on the same path, it will serve JSON
dawehner’s picture

Issue summary: View changes
dawehner’s picture

Wim Leers’s picture

dawehner’s picture

Title: [PP-1] EntityResource: GET/PATCH/DELETE to /node, but POST to /entity/node » EntityResource: GET/PATCH/DELETE to /node, but POST to /entity/node
Status: Postponed » Needs review

This should pass now.

Status: Needs review » Needs work

The last submitted patch, 50: rest-post-canonical-2293697-50.patch, failed testing.

Wim Leers’s picture

Status: Needs work » Needs review

Random migrate fail:

fail: [Other] Line 139 of core/modules/migrate_drupal_ui/src/Tests/MigrateUpgradeTestBase.php:

Retesting.

dawehner’s picture

I just checked page_manager and it indeed had a similar problem than node module. Given its an existing bug in contrib I'm not sure we should care about it: #2736385: Page Manager routing is overly complex and brittle: breaks REST, active trail, fallback pages

Wim Leers’s picture

#62: I think you wanted to post this on #2730497: REST Views override existing REST routes, not this issue?

dawehner’s picture

@Wim Leers
No, I'm actively reporting it here. There could be a chance that some people used page manager for /node, so when they do and we commit this patch, their sites would be broken.

Wim Leers’s picture

Ah, ok, thanks for the clarification!

Wim Leers’s picture

Issue summary: View changes
Status: Needs review » Needs work
Issue tags: -Needs issue summary update, -Needs change record +Needs tests

I rerolled this patch thrice:

  1. #35: straight reroll, rebased
  2. #37: added an initial BC layer, which @dawehner then completely redid in #46
  3. #50: address nitpicks

Clearly, I only played a tiny role in writing the actual patch. So I think it's fine for me to RTBC this issue.

I think the patch is ready. Still to be done: issue summary update & change record. I did both of those things (CR: https://www.drupal.org/node/2737401). I also updated the existing documentation (https://www.drupal.org/node/2098511/revisions/view/9709507/9730869). Let's get this in!

Except… there is no test coverage yet to prove that /entity/{entity_type} still works. So we need a small test proving that still works, then this is ready.

dawehner’s picture

Well, I still think we should consider to fix page_manager first. Just imagine /user etc. to be broken on actual sites. There is certainly a non null probability for it.

dawehner’s picture

Status: Needs work » Needs review
Issue tags: -Needs tests
FileSize
14.43 KB
5.15 KB

Good point, let's add the tests

dawehner’s picture

For me this seems to be ready to role, anyone wants to object ;)

Wim Leers’s picture

Status: Needs review » Reviewed & tested by the community

It looks perfect:

  1. +++ b/core/modules/rest/src/Tests/CreateTest.php
    @@ -120,10 +120,12 @@ public function testCreateEntityTest() {
    +      foreach ([NULL, '/entity/entity_test'] as $path) {
    

    Tests new URL and BC for POSTing entities of type entity_test .

  2. +++ b/core/modules/rest/src/Tests/CreateTest.php
    @@ -200,12 +201,14 @@ public function testCreateNode() {
    +      foreach ([NULL, '/entity/node'] as $path) {
    

    Tests new URL and BC for POSTing entities of type node .

  3. +++ b/core/modules/rest/src/Tests/CreateTest.php
    @@ -252,11 +255,13 @@ protected function testCreateComment() {
    +    foreach ([NULL, '/entity/comment'] as $path) {
    

    Tests new URL and BC for POSTing entities of type comment .

  4. +++ b/core/modules/rest/src/Tests/CreateTest.php
    @@ -298,12 +303,14 @@ public function testCreateUser() {
    +      foreach ([NULL, '/entity/user'] as $path) {
    

    Tests new URL and BC for POSTing entities of type user .

Yay, let's finally get this in :)

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 68: 2293697-15.patch, failed testing.

Wim Leers’s picture

It looks like this needs a reroll now that #2724823: EntityResource: read-only (GET) support for configuration entities has landed:

fail: [Other] Line 92 of core/modules/rest/src/Tests/ReadTest.php:
Response message is correct.
Value '{"message":"The \\u0022config_test\\u0022 parameter was not converted for the path \\u0022\\/entity\\/config_test\\/{config_test}\\u0022 (route name: \\u0022rest.entity.config_test.GET.hal_json\\u0022)"}' is identical to value '{"message":"The \\u0022config_test\\u0022 parameter was not converted for the path \\u0022\\/config_test\\/{config_test}\\u0022 (route name: \\u0022rest.entity.config_test.GET.hal_json\\u0022)"}'.

fail: [Other] Line 92 of core/modules/rest/src/Tests/ReadTest.php:
Response message is correct.
Value '{"message":"The \\u0022taxonomy_vocabulary\\u0022 parameter was not converted for the path \\u0022\\/entity\\/taxonomy_vocabulary\\/{taxonomy_vocabulary}\\u0022 (route name: \\u0022rest.entity.taxonomy_vocabulary.GET.hal_json\\u0022)"}' is identical to value '{"message":"The \\u0022taxonomy_vocabulary\\u0022 parameter was not converted for the path \\u0022\\/taxonomy_vocabulary\\/{taxonomy_vocabulary}\\u0022 (route name: \\u0022rest.entity.taxonomy_vocabulary.GET.hal_json\\u0022)"}'.

fail: [Other] Line 92 of core/modules/rest/src/Tests/ReadTest.php:
Response message is correct.
Value '{"message":"The \\u0022block\\u0022 parameter was not converted for the path \\u0022\\/entity\\/block\\/{block}\\u0022 (route name: \\u0022rest.entity.block.GET.hal_json\\u0022)"}' is identical to value '{"message":"The \\u0022block\\u0022 parameter was not converted for the path \\u0022\\/block\\/{block}\\u0022 (route name: \\u0022rest.entity.block.GET.hal_json\\u0022)"}'.

fail: [Other] Line 92 of core/modules/rest/src/Tests/ReadTest.php:
Response message is correct.
Value '{"message":"The \\u0022user_role\\u0022 parameter was not converted for the path \\u0022\\/entity\\/user_role\\/{user_role}\\u0022 (route name: \\u0022rest.entity.user_role.GET.hal_json\\u0022)"}' is identical to value '{"message":"The \\u0022user_role\\u0022 parameter was not converted for the path \\u0022\\/user_role\\/{user_role}\\u0022 (route name: \\u0022rest.entity.user_role.GET.hal_json\\u0022)"}'.

Those are specifically referring to the config entities that can now be POSTed. The path processor needs to only strip the leading /entity if the entity type has a canonical path.

dawehner’s picture

Status: Needs work » Needs review
FileSize
15.15 KB
738 bytes
Wim Leers’s picture

Status: Needs review » Reviewed & tested by the community

Wonderful, thanks!

alexpott’s picture

What does this mean for

Since most configuration entities do not canonical links they will be available at the path 'entity/[ENTITY_TYPE]/[ENTITY_ID]'.
For example to retrieve the tags vocabulary use.
curl --user admin:admin --request GET "http://drupal.d8/entity/taxonomy_vocabulary/tags?_format=json"

From #2724823: EntityResource: read-only (GET) support for configuration entities?

Wim Leers’s picture

As the patch shows: /<config entity type>/<entity ID>.

alexpott’s picture

@Wim Leers but doesn't that conflict with

The path processor needs to only strip the leading /entity if the entity type has a canonical path.

Wim Leers’s picture

Status: Reviewed & tested by the community » Needs review

Let me take a deeper look at this tomorrow, so I can formulate a proper answer.

dawehner’s picture

+++ b/core/modules/rest/src/Plugin/Deriver/EntityDeriver.php
@@ -73,8 +73,8 @@ public function getDerivativeDefinitions($base_plugin_definition) {
         $default_uris = array(
-          'canonical' => "/entity/$entity_type_id/" . '{' . $entity_type_id . '}',
-          'https://www.drupal.org/link-relations/create' => "/entity/$entity_type_id",
+          'canonical' => "/$entity_type_id/" . '{' . $entity_type_id . '}',
+          'https://www.drupal.org/link-relations/create' => "/$entity_type_id",
         );

I would indeed argue that we have an out of scope change here. The default canonical aka. fallback path for entities is /entity/{$entity_type_id}, so config entities are currently exposed under /entity/config_test/{config_test} for example.
We introduce a BC break in a way, or at least a change which is not part of the problem in most cases.

One could say \Drupal\rest\Tests\ReadTest::getReadUrl is a sign of a too abstracted test, as its not entirely obvious what is going on.

Status: Needs review » Needs work

The last submitted patch, 79: 2293697-79.patch, failed testing.

dawehner’s picture

Status: Needs work » Needs review
FileSize
14.24 KB
738 bytes

This was just a GET URL involved

Wim Leers’s picture

The patch in #73 and all prior patches changed the default canonical URI path in EntityResource. That URI path was used if the entity type didn't specify a specific canonical URI path. Technically it is indeed out of scope to do that, and from that POV #79 and #81 are correct.

However… in doing this, AFAICT this will start to break in #2300677: JSON:API POST/PATCH support for fully validatable config entities: because PathProcessorEntityResourceBC always strips the /entity prefix for POST requests, you won't be able to HTTP POST /entity/taxonomy_vocabulary, for example: the inbound path processor that this patch will always rewrite that to HTTP POST /taxonomy_vocabulary, which points to a non-existing path.

I'd argue that it is in the spirit of this patch to change all entity REST URIs to use /{entity_type}/{entity_id}, and therefore to change the default/fallback canonical entity resource URI paths. In which case the previous patch, i.e. the one in #73, is the appropriate one to commit.

In doing so, we'd have:

  1. consistent behavior across all entity types
  2. no breakage for content entity types without a canonical URI path (such as File)
  3. no breakage for config entity types when they gain the ability to be POSTed to the REST resource (i.e. in #2300677: JSON:API POST/PATCH support for fully validatable config entities
dawehner’s picture

Title: EntityResource: GET/PATCH/DELETE to /node, but POST to /entity/node » EntityResource: Use /{entity_type} for all REST routes

Technically it was decided in #2019123: Use the same canonical URI paths as for HTML routes to do so, so sure, let's rename the issue title then.

dawehner’s picture

We agreed on changing it

Wim Leers’s picture

Status: Needs review » Reviewed & tested by the community

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 84: 2293697-73.patch, failed testing.

Wim Leers’s picture

Status: Needs work » Reviewed & tested by the community

UpdatePathWithBrokenRoutingFilledTest failed. Let's see if it was a random fail or whether HEAD is broken.

dawehner’s picture

Yeah at that point in time basically every update test is failing randomly.

alexpott’s picture

Issue summary: View changes
alexpott’s picture

Status: Reviewed & tested by the community » Needs work
+++ b/core/modules/rest/src/PathProcessor/PathProcessorEntityResourceBC.php
@@ -0,0 +1,51 @@
+    if ($request->getMethod() === 'POST' && strpos($path, '/entity/') === 0) {

+++ b/core/modules/rest/src/Plugin/Deriver/EntityDeriver.php
@@ -73,8 +73,8 @@ public function getDerivativeDefinitions($base_plugin_definition) {
-          'canonical' => "/entity/$entity_type_id/" . '{' . $entity_type_id . '}',
-          'https://www.drupal.org/link-relations/create' => "/entity/$entity_type_id",
+          'canonical' => "/$entity_type_id/" . '{' . $entity_type_id . '}',
+          'https://www.drupal.org/link-relations/create' => "/$entity_type_id",

Now we've changed this issue to also change the default for canonical - we'd need to also fix GETs but that seems super risky.

dawehner’s picture

What @alexpott and myself talked about is the following:

  • Don't change the default, but rather change it for the entity types in core. This results in no BC break for other entity types
  • Change the BC layer to not just use the /entity prefix but also really just change /entity/\d+
  • Another idea I had in mind, store the regexes of those rest routes and use that in the path processor. This ensure that we just convert the examples, that has to be converted

All in all this would result in a more minimal BC break.

Wim Leers’s picture

#90: Damn, good point :/

#91:

This results in no BC break for other entity types

But we want custom/contrib entity types to also post to /<entity type>, not to /entity/<entity type>. We want that to still be consistent, so we should still do it for all entity types that do specify a canonical URL.

So, I agree with everything in #91, except for one thing: but rather change it for the entity types in core. — we shouldn't do that, we should do it for all entity types that do specify a canonical URL in my opinion.

dawehner’s picture

But we want custom/contrib entity types to also post to /, not to /entity/. We want that to still be consistent, so we should still do it for all entity types that do specify a canonical URL.

There is no reason that they cannot do that. They can add the link template. Risking BC when we don't have to, is IMHO an antipattern.

Wim Leers’s picture

They can add the link template.

Ahh! Now I understand.

So they'd each have to do something like this:

/**
 * Defines the node entity class.
 *
 * @ContentEntityType(
 *   id = "node",
 *   label = @Translation("Content"),
…
 *   links = {
 *     "canonical" = "/node/{node}",
 *     "delete-form" = "/node/{node}/delete",
 *     "edit-form" = "/node/{node}/edit",
 *     "version-history" = "/node/{node}/revisions",
 *     "revision" = "/node/{node}/revisions/{node_revision}/view",
… 
 *     "https://www.drupal.org/link-relations/create" = "/node",
 *   }
 * )
 */

(Look at that very last line in the annotation.)

That makes sense.

+1

dawehner’s picture

@Wim Leers Exactly, do you think less risk is better here?

Wim Leers’s picture

Yes, +1.

Wim Leers’s picture

Actually, @EclipseGc made an excellent remark yesterday in personal chat. Why are we not simply using the collection link relation? Creating a new entity of a certain type is the same as POSTing something to a collection. That makes perfect sense. It'd also mean we can deprecate the silly https://www.drupal.org/link-relations/create link relationship. https://en.wikipedia.org/wiki/Representational_state_transfer#Relationsh... also explicitly mentions this.

So I did some digging. Apparently #2019123: Use the same canonical URI paths as for HTML routes introduced the https://www.drupal.org/link-relations/create link relationship. Despite that issue being done by resident REST experts @klausi and @Crell, there were zero mentions of collection (well, there were mentions of that particular word in the context of a route collection, but the link relationship was not mentioned once).

Now, the funny thing is that we have 14 entity types in core that specify a collection relationship. But node isn't one of them. Nor is comment.

So, it looks like in order to do what is really right, we'll first have to fix the link relations on many entity types.

Thoughts?

Crell’s picture

The "every resource type has one definitive collection and you POST to that to create something" is not, AFAIK, part of REST style as defined by Fielding. It's a common pattern seen in demo applications and other cases where "All the Thingies" is the only reasonable collection of Thingies, but that's not at all universal. In fact, I have built systems for clients where we had /thingie/1/stuff/4 as a "stuff" resource, and the "collection" was therefore /thingie/1/stuff, and that's where you'd POST. There was also /thingie/8/stuff if you wanted to add things to THAT collection. (In this case, it was a collection of stuff that belonged to that particular thingie.)

Thus, a blanket statement that all entities are primarily a part of the "all the thingies" collection, and that is their primary existence is, I would argue, incorrect. That's why we didn't do that. Arguably we could make the assumption that all comments are first and foremost part of the node/X/comment collection (or whatever other base entity they are on, since in D8 comments are not node-specific), but I don't think any such assumption could be made for nodes.

Wim, you don't mention which those 14 entities are that have a collection relationship, but I'll venture a guess that many/most are config entities, or other entities with very low cardinality and relationship complexity. In those cases the simplified default assumption probably dos hold. But that's not the universal REST Way, just a common degenerate case, and so not an assumption we should bake into the whole system.

That is, "Creating a new entity of a certain type is the same as POSTing something to a collection" is only true sometimes. It is not a universal.

tstoeckler’s picture

That might be complete bogus in a REST world, but we currently use collection routes for entity list builders. The fact that not very many entity types actually provide link templates for the collection link templates is owed to the fact that we are not capable of auto-generating a list-builder-route yet (!). Once we can do that, we will add collection link templates, but they will point to paths like /admin/content and /admin/config/user-interface/shortcut. While - again - that might make the stomache of someone coming from a REST POV twist, I don't think that's something we can change at this point.

Might also be that this is completely beside the issue and I'm just rambling, in which case sorry and carry on, but the mention of the collection relation above made me stutter for a moment.

Wim Leers’s picture

#98: thanks a lot for that comment, that was very interesting!

#99: indeed, many of the collection routes are admin-UI-specific URLs, which makes them… less than ideal for REST purposes. However, I'm not sure I understand your conclusion: are you saying you're agreeing with #97, or that you were frustrated by the explanation in #98, or …?

EclipseGc’s picture

@Crell

So, while I don't have a problem with your basic argument, your example does nothing to convince me. That's still defining the collection of the stuff entity as /thingie/{thingie}/stuff. It is then perfectly logical that posting to that will make a stuff entity related to the thingie in question. This is EXACTLY how I'd want comments (and taxonomy terms) to behave. Of course none of this has to be forcefully thrust upon all entity types, but as a guiding principle, I think it's a pretty good (and common) one. No?

Eclipse

Crell’s picture

Well, it's true in many cases, certainly. But to take comments, we support comments on nodes, on users, on commerce products, etc. Technically, we even support multiple comment fields/threads on a single entity. (I think? I've not actually tried and I don't know why you would, but isn't that a side effect of comment-config-as-field?) That means simply defining the comment "collection" route as /node/{node}/comments isn't sufficient, because comments would also make sense at /user/{user}/comments, /node/{node}/internal_comments (alternate field), /commerce/product/{product}/comments, etc.

That is, simply standardizing all entities to a "collection" route, in their annotation, of /{entity_type} is not going to work. That would be true in some cases, perhaps a majority, but would break badly in many. We would need to at least allow for collection to be defined contextually at runtime instead.

dawehner’s picture

While I think the discussion around the collection link relation is worth to do, I think this issue is about a bug in REST itself and its inconsistency. Fixing that is a small iteration step, vs. what we discuss which is a big step compared to it. Its totally worth to do, but it should be its own thing.

Regarding the collections, I think those collections should be created, but in order to design properly, one needs domain knowledge, rest module cannot have. REST module made the tradeoff early that its primarily a generic export all the things mechanism, rather something well tuned like a RESTAPI of github for example.

tstoeckler’s picture

I only brought the collections thing up because - per this issue's title - I thought the intent was to make the "collection" link in the entity annotation point to e.g. "/node". That would be a pretty big conceptual (and API) change, which I tried to explain above. If that is in fact not the case, then this can be ignored. I'm not sure reading the latest comments what the intent of the issue is, TBH.

Wim Leers’s picture

So let's get this issue back on track! What still needs to happen for this issue to go back to RTBC?

  1. In #90, Alex Pott un-RTBC'd it.
  2. In #91–#96, the necessary changes were discussed and fully agreed upon.
  3. Then, in #97, I posted a comment that basically proposed a very different direction that @EclipseGc told me about: using the collection link relationship's URI.
  4. In #98, Crell explained that EclipseGc rationale is indeed very common, but not universally applicable. And hence it is an assumption we should not bake into the system.
  5. in #101 and #102 Crell and EclipseGc continue the discussion.
  6. In #103, dawehner argues that assuming that we can use collection, then it'd still be inappropriate to do so here, because it requires many more changes.

So, basically, #97 through #104 was a big, distracting What if? that would lead to a different direction for the patch. And it'd be very valuable, if it turns out to be possible.
But this issue is just about fixing the WTF that is GET /node/42, PATCH /node/42, DELETE /node/42 versus POST /entity/node. That should just be POST /node (every person who's used REST so far has found that unintuitive/strange). That's all this is about. And that's something we can do without the even bigger changes proposed in #97–#104.

Therefore let us go back to #90 + #91–#96, where we were so close to a patch that resolves this big REST Developer Experience WTF.


This implements everything discussed in #90#97. CR updated as well: https://www.drupal.org/node/2737401/revisions/view/9730871/9919603.

I think this idea in #91 would be a splendid minor performance improvement for a follow-up:

Another idea I had in mind, store the regexes of those rest routes and use that in the path processor. This ensure that we just convert the examples, that has to be converted

But for now, I've gone with the simplest possible approach that works, so that this can still go in before feature freeze next week.

Wim Leers’s picture

Title: EntityResource: Use /{entity_type} for all REST routes » EntityResource POST routes all use the confusing default in <8.2: use entity types' https://www.drupal.org/link-relations/create link template if available

Already updated the patch, CR and IS to reflect the new direction. Now also updating the title.

This is so much better. Thanks, @alexpott and @dawehner!

Status: Needs review » Needs work

The last submitted patch, 105: 2293697-105.patch, failed testing.

dawehner’s picture

  1. +++ b/core/modules/rest/src/PathProcessor/PathProcessorEntityResourceBC.php
    @@ -0,0 +1,51 @@
    +    if ($request->getMethod() === 'POST' && strpos($path, '/entity/') === 0) {
    +      $parts = explode('/', $path);
    +      $entity_type = array_pop($parts);
    

    Can we ensure to just match on /entity/{entity_type} and nothing more?

  2. +++ b/core/modules/taxonomy/src/Entity/Term.php
    @@ -46,6 +46,7 @@
    + *     "https://www.drupal.org/link-relations/create" = "/taxonomy_term",
    

    IMHO this path is a bit weird. Don't we want /taxonomy/term instead?

Wim Leers’s picture

  1. That's what the code after that does? Hm, I see it's not strict enough. Let me fix that.
  2. Because it's GET|PATCH|DELETE /taxonomy_term/42 already. It'd be much more confusing to have GET /taxonomy_term/42, but POST /taxonomy/term
dawehner’s picture

Hm, I see it's not strict enough. Let me fix that.

Yeah this was kinda the point alexpott and I talked about in #90 and #91

Well, GET is certainly on "/taxonomy/term/{taxonomy_term}" but yeah we some some inconsistency there. nevermind.

dawehner’s picture

So yeah please ignore me little rambling there :)

dawehner’s picture

These test failures are kind of interesting to be honest.
They are basically really related with #2113345: Define a mechanism for custom link relationships as in we have, names for our link relations, and external URLS where they are defined.

[28-Jul-2016 23:07:40 Australia/Sydney] Uncaught PHP Exception Symfony\Component\Routing\Exception\RouteNotFoundException: "Route "entity.node.https://www.drupal.org/link_relations/create" does not exist." at /var/www/html/core/lib/Drupal/Core/Routing/RouteProvider.php line 187

In case we would have #2113345: Define a mechanism for custom link relationships in, there would be also a way to use "create" in the link templates, but then still expose the right link relation to the public.

tedbow’s picture

Is possible to fix without #2113345: Define a mechanism for custom link relationships?

Classes like \Drupal\node\Controller\NodeViewController assume call \Drupal\Core\Entity\Entity::url which will call \Drupal\Core\Entity\Entity::toUrl

Entity::toUrl assumes it can make a route name with $rel.

$route_name = "entity.{$this->entityTypeId}." . str_replace(array('-', 'drupal:'), array('_', ''), $rel);
      $uri = new Url($route_name, $route_parameters);

That is why you get the error in #112.

Also on a related note should we actually be updating Entity class files like Node.php?

Isn't
"https://www.drupal.org/link-relations/create" = "/node",
Only valid if the REST module enabled or maybe I am missing something?

Should we be using hook_entity_type_alter to add this links to the entity definition? This would probably fix a lot of the test because they won't turn on the REST module but it won't fix the underlying problem.

dawehner’s picture

Status: Needs work » Postponed

Yeah let's be honest, this gives us the abstractions we need

dawehner’s picture

Title: EntityResource POST routes all use the confusing default in <8.2: use entity types' https://www.drupal.org/link-relations/create link template if available » [pp-1] EntityResource POST routes all use the confusing default in <8.2: use entity types' https://www.drupal.org/link-relations/create link template if available

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.0-beta1 was released on August 3, 2016, which means new developments and disruptive changes should now be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Wim Leers’s picture

Title: [pp-1] EntityResource POST routes all use the confusing default in <8.2: use entity types' https://www.drupal.org/link-relations/create link template if available » EntityResource POST routes all use the confusing default in <8.2: use entity types' https://www.drupal.org/link-relations/create link template if available
Version: 8.4.x-dev » 8.3.x-dev
Status: Postponed » Needs work

#2113345: Define a mechanism for custom link relationships is FINALLY in, so now this can continue! We should still try to get this in 8.3.

I will reroll this first thing in the morning unless somebody beats me to it.

Wim Leers’s picture

Status: Needs work » Needs review
FileSize
7.03 KB
11.21 KB

First, a straight reroll.

#2737719: EntityResource: Provide comprehensive test coverage: for every entity type, every format, every method completely changed the test coverage, and thus those portions no longer apply: core/modules/rest/src/Tests/CreateTest.php and core/modules/rest/src/Tests/ReadTest.php no longer exist. This means we must still do the work to restore the removed test coverage! (I've attached an interdiff that shows which parts of the patch I removed. It's not actually an interdiff, it's the portion of #105 that I've literally cut away, because it cannot apply at all anymore.)

I expect this to have a lot of fails.

tstoeckler’s picture

+++ b/core/modules/rest/src/PathProcessor/PathProcessorEntityResourceBC.php
@@ -0,0 +1,51 @@
+      if ($this->entityTypeManager->getDefinition($entity_type)->hasLinkTemplate('https://www.drupal.org/link-relations/create')) {
+        // Return the path minus the leading '/entity'.
+        return substr($path, 7);

+++ b/core/modules/taxonomy/src/Entity/Term.php
@@ -46,6 +46,7 @@
+ *     "https://www.drupal.org/link-relations/create" = "/taxonomy_term",

Couldn't we return $entity_type->getLinkTemplate(...) instead of always using the entity type ID? I.e. for taxonomy terms maybe we want to use /taxonomy/term?

Edit: I see that that has been discussed in #108 and following, but it still seems that in general this should be configurable and we should use the provided link template?

Status: Needs review » Needs work

The last submitted patch, 119: 2293697-119.patch, failed testing.

xjm’s picture

Version: 8.3.x-dev » 8.4.x-dev
Wim Leers’s picture

#120: I think you're right :) That definitely needs to be done before this can be RTBC. But first, bigger worries: getting this patch to green. Details about the trickiness to follow…

Wim Leers’s picture

Status: Needs work » Needs review
Related issues: +#2113345: Define a mechanism for custom link relationships
FileSize
10.32 KB
8.35 KB

#2113345: Define a mechanism for custom link relationships added link relations, but did so with https://drupal.org/link-relations/… URIs instead of https://www.drupal.org/link-relations/… which was used in this patch. Rather than modifying core.link_relation_types.yml, I think it's better to modify this patch.

Not only that, but it's probably better/more accurate to use the key specified in core.link_type_relations.yml: i.e. use create, not https://drupal.org/link-relations/create. Just like entity type annotations.


This made me realize: then how can you discern registered link relation types versus extension link relation types?! We should use identify them by their name and URI respectively. That'd match the spec. So we should continue to do it like this patch is already doing: using URIs!

However… that won't work because Drupal 8 shipped with violations of this principle already, and we can't change that because it'd break BC. We should have done #2113345: Define a mechanism for custom link relationships before D8 shipped, then we wouldn't have made this mistake. For example for Node in HEAD:

 *   links = {
 *     "canonical" = "/node/{node}",
 *     "delete-form" = "/node/{node}/delete",
 *     "edit-form" = "/node/{node}/edit",
 *     "version-history" = "/node/{node}/revisions",
 *     "revision" = "/node/{node}/revisions/{node_revision}/view",
 *   }

Of those, only canonical, edit-form and version-history are registered link relation types (i.e. registered with and accepted by the IANA). So that means delete-form and revision are already extension link relation types (i.e. Drupal-specific).

(This also means that if the IANA accepts new link relation types that happen to use the same names as the ones Drupal used, but with a different meaning, that we will be forced to break BC. For D9, we should change the entity type links annotation to use URIs as keys for extension link relation types, and we should also make link_relation_types.yml use URIs as keys for those.)

So the existing entity type annotations (and associated logic) requires us to not use URIs. But OTOH the REST module's logic requires us to use URIs. There's no way to reconcile these. We must break one of the two!

I think there is one of those that is clearly more important (and would cause a bigger disruption): the entity type annotation. Very few @RestResource plugins have ever figured out how to define a custom deriver. Plus we can add a BC layer in \Drupal\rest\Plugin\ResourceBase::routes(), which means we can actually guarantee zero disruption. Whew!


Ironically, this (the Node entity type annotation cited above) was introduced by #1970360: Entities should define URI templates and standard links, which itself was all about using link relation types + associated link templates for REST… that enabled #2019123: Use the same canonical URI paths as for HTML routes, which is what made the REST module actually use entity type-defined link templates. It already pointed to #2113345: Define a mechanism for custom link relationships as the place to settle on a final solution, but said it should not be blocked on that issue. That was a huge mistake. That's why we have this mess. That's why this issue is still not solved, more than 3.5 years after #2019123 was opened.
In fact, @linclark predicted that the logic in the REST module would have to change depending on how #2113345: Define a mechanism for custom link relationships would be solved. See her comment at #2019123-43: Use the same canonical URI paths as for HTML routes. Also, it's that same issue (#2019123) that made bold claims about how much better REST URLs became (see the change record), but it also forgot to mention the fact that it only did so for GET/PATCH/DELETE, it did not make the POST case better. Which is what we're still trying to solve here, almost 3 years after that was committed.


Therefore, we have but one choice:

+++ b/core/modules/aggregator/src/Entity/Feed.php
@@ -33,6 +33,7 @@
+ *     "https://www.drupal.org/link-relations/create" = "/aggregator/sources",

+++ b/core/modules/block_content/src/Entity/BlockContent.php
@@ -41,6 +41,7 @@
+ *     "https://www.drupal.org/link-relations/create" = "/block",

Change hunks like these to use "create" = "…" instead. Did that.

This causes some disruption, but we can mitigate it completely by doing Plus we can add a BC layer in \Drupal\rest\Plugin\ResourceBase::routes(), which means we can actually guarantee zero disruption. — did that too.

The last submitted patch, 124: 2293697-124.patch, failed testing.

Wim Leers’s picture

As you can tell, the test results of #124 are littered with PHP errors like this:

Uncaught PHP Exception Symfony\Component\Routing\Exception\RouteNotFoundException: "Route "entity.node.create" does not exist." at /var/www/html/core/lib/Drupal/Core/Routing/RouteProvider.php line 187

This is because \Drupal\node\Controller\NodeViewController::view() (soon hopefully EntityViewController — see #2282029: Automate link relationship headers for all entity types, not just nodes) and \Drupal\rest\Plugin\rest\resource\EntityResource::addLinkHeaders() call $node->toUrl('create') (which is correct btw). That in turn tries to generate the URL for the entity.node.create route. Which… fails. Because it doesn't exist.

It doesn't exist, because despite create being a link relation type specifically for REST, \Drupal\rest\Plugin\ResourceBase::routes() always wants to create routes with names like rest.entity.node.POST.

So this breaks both REST responses and HTML responses for /node/NID, with the fatal error mentioned above.


We could make all code that iterates over link templates should support the fact that a route may not be defined. Then the create link would never be listed, but at least it wouldn't cause fatal errors.

Let's at least do that for now, that should reduce the number of failures.

Wim Leers’s picture

FileSize
7.18 KB
20.28 KB

So #126 was not a full solution. We are not yet able to generate create link headers, because the necessary route does not exist.


Symfony does not support the concept of "route aliases" or "route pointers". We'd have to have rest.entity.node.POST and entity.node.create. And they'd have to have identical route definitions. That's the only way to make two route names point to the same route.

We can't drop rest.entity.node.POST either, doing so might break BC. Nor can we drop entity.node.create, because that would break both \Drupal\Core\Entity\Entity::toUrl() and the entity type link templates API.


Letting \Drupal\rest\Plugin\rest\resource\EntityResource::routes() generate a entity.node.create route with such a duplicate definition doesn't work, because \Drupal\rest\Routing\ResourceRoutes::alterRoutes() prefixes all routes with 'rest'.
I can make that work by also modifying ResourceRoutes::alterRoutes() to no longer set the rest. prefix on route names, and just setting that prefix in EntityResource and DBLogResource. But this would break BC for other REST resources — their route names would change.


That almost leaves me with only one choice: a \Drupal\Core\Routing\RoutingEvents::DYNAMIC event subscriber that iterates over all RestResourceConfig entities, looks which ones are for the EntityResource @RestResource plugin, and adds a entity.<entity type>.create route with the same definition as the rest.entity.<entity type>.POST.
But this feels quite strange IMO.

Modeled after \Drupal\Core\EventSubscriber\EntityRouteProviderSubscriber.

And UGH… in the process I noticed that \Drupal\rest\Routing\ResourceRoutes is using RoutingEvents::ALTER, but it really should have been using RoutingEvents::DYNAMIC! To be fair, that was added in #2158571: Routes added in RouteSubscribers cannot be altered, long after this class was created. But irony oh irony, this very class was mentioned in #2158571 many times as an example of what should use DYNAMIC, but then it didn't do that. So, fixing that too.

Wim Leers’s picture

FileSize
1.65 KB
20.25 KB

Now addressing #120:

Couldn't we return $entity_type->getLinkTemplate(...) instead of always using the entity type ID? I.e. for taxonomy terms maybe we want to use /taxonomy/term?

Fixed both of those things.

Wim Leers’s picture

FileSize
1.39 KB
21.15 KB

Adding back test coverage that was lost in #119. Thanks to the overhauled test coverage, adding test coverage has just become trivial.

Wim Leers’s picture

FileSize
6.31 KB
21.44 KB

Round of self-review.

  1. +++ b/core/modules/aggregator/src/Entity/Feed.php
    @@ -33,6 +33,7 @@
    + *     "create" = "/aggregator/sources",
    

    This is a config entity. We shouldn't add this link template for config entities yet.

    (Config entities currently only support GET.)

  2. +++ b/core/modules/rest/src/EventSubscriber/EntityResourcePostRouteSubscriber.php
    @@ -0,0 +1,74 @@
    +      // We only care about REST resource config entities for the
    

    s/for/that use/

  3. +++ b/core/modules/rest/src/PathProcessor/PathProcessorEntityResourceBC.php
    @@ -0,0 +1,51 @@
    +      // Only remove the '/entity' prefix if we have a matching entity type
    +      // following it. Otherwise, this is a route other than the one for
    +      // \Drupal\rest\Plugin\rest\resource\EntityResource, and hence we don't
    +      // want to change anything.
    +      if ($this->entityTypeManager->getDefinition($entity_type)->getLinkTemplate('create')) {
    +        // Return the path minus the leading '/entity'.
    +        return substr($path, 7);
    +      }
    

    The comment doesn't match the logic. And actually, the logic is very wrong too.

    My changes in #128 were completely wrong. It's been a long day :(

    This means that e.g. for taxonomy terms, this will fail.

  4. +++ b/core/modules/rest/src/Plugin/ResourceBase.php
    @@ -100,8 +100,11 @@ public function routes() {
    -    $create_path = isset($definition['uri_paths']['https://www.drupal.org/link-relations/create']) ? $definition['uri_paths']['https://www.drupal.org/link-relations/create'] : '/' . strtr($this->pluginId, ':', '/');
    -
    +    $create_path = isset($definition['uri_paths']['create']) ? $definition['uri_paths']['create'] : '/' . strtr($this->pluginId, ':', '/');
    

    We shouldn't change whitespace.

  5. +++ b/core/modules/rest/src/Plugin/ResourceBase.php
    @@ -100,8 +100,11 @@ public function routes() {
    +    // BC.
    

    This needs a better comment.

  6. +++ b/core/modules/rest/src/Plugin/rest/resource/EntityResource.php
    @@ -369,9 +370,19 @@ public function calculateDependencies() {
    -          ->toString(TRUE);
    +
    +        // It's not guaranteed that every link relation type also has a
    

    Unnecessary whitespace.

  7. \Drupal\rest\Plugin\rest\resource\EntityResource still has:
     *   uri_paths = {
     *     "canonical" = "/entity/{entity_type}/{entity}",
     *     "https://www.drupal.org/link-relations/create" = "/entity/{entity_type}"
     *   }
    

    That should become create.

  8. Should PathProcessorEntityResourceBC be renamed to PathProcessorEntityResourcePostRouteBC?

Addressed everything except the last point.

The last submitted patch, 126: 2293697-126.patch, failed testing.

Wim Leers’s picture

Issue summary: View changes

CR updated.

IS updated.

The last submitted patch, 127: 2293697-127.patch, failed testing.

The last submitted patch, 128: 2293697-128.patch, failed testing.

The last submitted patch, 129: 2293697-129.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 130: 2293697-130.patch, failed testing.

Wim Leers’s picture

Sigh, complete DrupalCI failure. We'll have to wait for DrupalCI to work again before we can retest.

#127 through #130 will need to be retested.

tedbow’s picture

Status: Needs work » Needs review
FileSize
20.38 KB
1.49 KB
  1. +++ b/core/modules/comment/src/Entity/Comment.php
    index fce50e1..ba73ba6 100644
    --- a/core/modules/node/src/Controller/NodeViewController.php
    
    +++ b/core/modules/node/src/Controller/NodeViewController.php
    @@ -56,6 +57,17 @@ public function view(EntityInterface $node, $view_mode = 'full', $langcode = NUL
    +      try {
    +        $generated_url = $url->toString();
    +      }
    +      catch (RouteNotFoundException $e) {
    +        continue;
    +      }
    +
    

    Is it preferable here to catch an exception rather than check the route exists?
    something like:

    if (!$url->isRouted() || !\Drupal::service('router.route_provider')->getRoutesByNames([$url->getRouteName()])) {
            continue;
          }

    Probably not but thought I would mention it.

  2. +++ b/core/modules/rest/src/EventSubscriber/EntityResourcePostRouteSubscriber.php
    @@ -0,0 +1,74 @@
    +      $rest_post_route = $route_collection->get($rest_post_route_name);
    +
    +      // Create a route for the 'create' link relation type for this entity type
    +      // that uses the same route definition as the REST 'POST' route which use
    +      // that entity type.
    +      // @see \Drupal\Core\Entity\Entity::toUrl()
    +      $entity_create_route_name = "entity.$entity_type_id.create";
    +      $route_collection->add($entity_create_route_name, $rest_post_route);
    

    If a particular entity resource is configured not to enabled POST $rest_post_route will be null and then a fatal error for adding the collection.

    I hit this b/c the site I was debugging had 'file' entity type setup this way.

    Fixed in attached patch.

Wim Leers’s picture

Wim Leers’s picture

#138.2: HAH! Yes, thanks :)

#138.1: The current approach works for any URL, yours requires more logic, and ends up doing the same thing. So not a fan of the approach you mentioned at first sight, but also not opposed.

Wim Leers’s picture

Title: EntityResource POST routes all use the confusing default in <8.2: use entity types' https://www.drupal.org/link-relations/create link template if available » EntityResource POST routes all use the confusing default in <8.3: use entity types' https://www.drupal.org/link-relations/create link template if available
tedbow’s picture

@Wim Leers for #138.2 that sounds like it is fine already the way it is.

Status: Needs review » Needs work

The last submitted patch, 138: 2293697-131.patch, failed testing.

dawehner’s picture

  1. +++ b/core/modules/node/src/Controller/NodeViewController.php
    @@ -56,6 +57,17 @@ public function view(EntityInterface $node, $view_mode = 'full', $langcode = NUL
    +      try {
    +        $generated_url = $url->toString();
    +      }
    +      catch (RouteNotFoundException $e) {
    +        continue;
    +      }
    

    Another possible exception could be a messing parameter in the generator URL, like when the relation type needs more than one parameter. \Symfony\Component\Routing\Exception\MissingMandatoryParametersException would be thrown in that case.

  2. +++ b/core/modules/rest/src/Routing/ResourceRoutes.php
    @@ -54,18 +56,18 @@ public function __construct(ResourcePluginManager $manager, EntityTypeManagerInt
    -  protected function alterRoutes(RouteCollection $collection) {
    +  public function onDynamicRouteEvent(RouteBuildEvent $event) {
    

    +1

Wim Leers’s picture

#144.1: Right, currently there are no create link templates that use parameters, but that's indeed possible. Fixing that…
Although hm, no, that's not possible. Look at \Drupal\Core\Entity\Entity::toUrl(). It automatically retrieves the necessary route parameters using \Drupal\Core\Entity\Entity::urlRouteParameters(). That already works for all other link relation types. So it will work fine for the create link relation type too. So the code is fine as-is.

Wim Leers’s picture

Status: Needs work » Needs review
FileSize
22.92 KB
1.5 KB

Apparently it's taxonomy_page_attachments_alter() that's adding link relation headers for the canonical Term route! We'll need to update that logic just like we updated NodeViewController's.

That will fix some failures.

The last submitted patch, 146: 2293697-146.patch, failed testing.

Wim Leers’s picture

FileSize
593 bytes
23.48 KB

There are a bunch of these failures:

Uncaught PHP Exception RuntimeException: "Callable "Drupal\Core\Access\CsrfRequestHeaderAccessCheck::access" requires a value for the "$request" argument." at /var/www/html/core/lib/Drupal/Component/Utility/ArgumentsResolver.php line 142

The root cause is simple: that service needs the request object (hence that error), but it's not tagged correctly.

Status: Needs review » Needs work

The last submitted patch, 148: 2293697-148.patch, failed testing.

Wim Leers’s picture

Status: Needs work » Needs review

100% of the remaining failures is in User entity type REST tests.

Why? Because thanks to #2293697: EntityResource POST routes all use the confusing default: use entity types' https://www.drupal.org/link-relations/create link template if available, we have comprehensive test coverage, including of edge cases. The failing assertions are asserting that we're getting an error response when POSTing to /user. But with this patch applied, we're not getting a 4xx response, we're getting a 200! Why? Because /user matches the user.page route:

user.page:
  path: '/user'
  defaults:
    _controller: '\Drupal\user\Controller\UserController::userPage'
    _title: 'My account'
  requirements:
    _user_is_logged_in: 'TRUE'

… which requires the user to be logged in. If the user is not logged in, one is redirected automatically to /user/login. So, Guzzle simply follows this redirect and then repeats the same POST request, but this time to /user/login.

This then matches three routes:

  1. user.login (/user/login), which has no _format or _content_type_format requirements
  2. user.login.http (/user/login), which has only a _format requirement
  3. user.entity.canonical (/user/{user}), which has no _format or _content_type_format requirements

The second one is filtered away by \Drupal\Core\Routing\MethodFilter during route matching. That leaves the first and the third. The first one is selected and since there's no actual request body (i.e. $_POST is empty), UserLoginForm happily returns the form (since an empty request body does not trigger form submission) and hence it results in a 200 response.


Now, it may be easy to blame the form system for sending a 200 response to an empty POST request. But the actual problem lies much earlier: in the redirect. It's the redirect that's causing all these problems. The redirect should never have happened! The redirect should only happen for HTML requests. What identifies HTML requests, is the absence of a ?_format string AND a Content-Type request header that is either:

  • absent (if GET request)
  • set to application/x-www-form-urlencoded
  • set to multipart/form-data

In other words: our pre-D8 days are coming to haunt us, the decision to let D7 menu callbacks become _controller callbacks (first _content) that return render arrays without needing to specify any additional requirements, is actually the problem here.

Changing the defaults (i.e. _content_type_format) set for such routes (see \Drupal\Core\EventSubscriber\RouteMethodSubscriber) is probably a bad idea; it's likely it'll cause some disruption. Plus, multipart/form-data is actually also very commonly used for REST, i.e. for non-HTML routes. So we can't assign that anyway.

I see only two solutions:

  1. Change the way EntityResourceTestBase uses Guzzle: make it no longer follow redirects. The 200 will become a 301. This is something we should probably do anyway.

    But it doesn't solve the actual problem: the redirection does not make sense.

  2. changing
    user.page:
      path: '/user'
      defaults:
        _controller: '\Drupal\user\Controller\UserController::userPage'
        _title: 'My account'
      requirements:
        _user_is_logged_in: 'TRUE'
    

    to:

    user.page:
      path: '/user'
      defaults:
        _controller: '\Drupal\user\Controller\UserController::userPage'
        _title: 'My account'
      requirements:
        _method: 'GET'
        _user_is_logged_in: 'TRUE'
    

    This will reject any POST requests for that route, which is fine, because it's a pure convenience route anyway, that always redirects (to either /user/login or to /user/{user}, depending on whether you're logged in or not).

    The current 200 response (well, the 301 redirect, then 200) will become a 405 response (method not allowed).

    But it doesn't solve the problem generically, it solves the problem only for this particular entity type, because it happens to have this particular route.

So I implemented this solution by implementing both parts. Part 1 is not necessary but helps prevent surprises. Part 2 is essential. It does mean that there is some special casing in EntityResourceBase for User, but it's very very limited.

Wim Leers’s picture

FileSize
3.46 KB
26.41 KB

WTF d.o. I pressed "upload", and you submitted it.

Here are then the files that belong to #150.

Wim Leers’s picture

FileSize
1000 bytes
26.49 KB

Tests were able to get further in the test scenario, but they got stuck on the final bit when posting users.

Here's the last necessary fix, to prevent those 422 responses (due to Unprocessable Entity: validation failed. name: The username Dramallama ic28G5Or is already taken) and get them to be 201 responses again.

Wim Leers’s picture

In another round of self-review, I can find only one nit:

+++ b/core/modules/rest/rest.services.yml
@@ -35,8 +35,23 @@ services:
+      - { name: path_processor_inbound }
+

Unnecessary blank line.


Hopefully this is now getting close to RTBC :)

The last submitted patch, 151: 2293697-150.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 152: 2293697-152.patch, failed testing.

Wim Leers’s picture

Assigned: Unassigned » Wim Leers

The two remaining tests are both Contact-related. Reproduced.

Wim Leers’s picture

Status: Needs work » Needs review
FileSize
4.28 KB
30.51 KB

So, this test is failing in ContactSidewideTest: $this->updateContactForm($id, $label = $this->randomMachineName(16), $recipients_str = implode(',', array($recipients[0], $recipients[1])), $reply = $this->randomMachineName(30), FALSE, 'Your message has been sent.', '/user');. Note that it's setting the redirect URL to /user. So now it's validating the /user URL using the PathValidator. And remember that in #150/#151, we configured that route to only allow the GET method to be used.


The root cause: PathValidator reuses the current request. Hence it inherits the POST method if PathValidator is called in a form's submit handler. And hence \Drupal\Core\Render\Element\PathElement::validateMatchedPath() now complains about /user, because of the changes in #150/#151.

Specifically that method calls:

  • \Drupal\Core\Path\PathValidator::getUrlIfValid(), which calls
  • \Drupal\Core\Path\PathValidator::getUrl(), which does
        $request = Request::create('/' . $path);
        $attributes = $this->getPathAttributes($path, $request, $access_check);
    
  • \Drupal\Core\Path\PathValidator::getPathAttributes() calls \Drupal\Core\Routing\Router::match(), which calls
  • \Drupal\Core\Routing\Router::matchRequest(), which calls
  • \Symfony\Component\Routing\Matcher\UrlMatcher::matchCollection(), which does NOT receive the request object, and instead retrieves the method like this:
    $method = $this->context->getMethod()
    

    i.e. it reads the request method from the global request context, and hence incorrectly concludes that it's matching a collection for a POST request!

    We need to make sure that we've updated the global request context object (the router.request_context service, actually) to use our fake request object that we use purely for validation purposes.

    Ideally, we should be able to just push our fake request onto the request stack… but sadly Symfony is incredibly poorly architected not taking care of that automatically, leaving us with the request stack and request context getting out of sync… So the solution is to not push onto the request stack, but rather to get the current request from the request stack, set our fake request on the request context, and then restore it onto the request context (because yay, the request context service/object has no way to read all of its information, so we can't read from it to then restore it).

Whew. This is a great showcase of why I proposed https://events.drupal.org/baltimore2017/sessions/maintainability-only-co....

dawehner’s picture

Wim Leers’s picture

#158: THANK YOU. I was dreading writing test coverage for this. And it really belongs in a separate issue. Very glad it already exists, and is already RTBC!

Status: Needs review » Needs work

The last submitted patch, 157: 2293697-157.patch, failed testing.

dawehner’s picture

Although hm, no, that's not possible

Oh yeah fair. When we run into the problem it would be a problem everywhere else?

  1. +++ b/core/core.services.yml
    @@ -1143,7 +1143,7 @@ services:
         class: Drupal\Core\Access\CsrfRequestHeaderAccessCheck
         arguments: ['@session_configuration', '@csrf_token']
         tags:
    -      - { name: access_check }
    +      - { name: access_check, needs_incoming_request: TRUE }
       maintenance_mode:
         class: Drupal\Core\Site\MaintenanceMode
    

    +1

  2. +++ b/core/modules/rest/tests/src/Functional/EntityResource/EntityResourceTestBase.php
    @@ -497,7 +497,7 @@ public function testPost() {
         if ($has_canonical_url) {
    -      $this->assertSame(404, $response->getStatusCode());
    +      $this->assertSame(static::$entityTypeId === 'user' ? 405 : 404, $response->getStatusCode());
    
    @@ -510,7 +510,12 @@ public function testPost() {
    +    if (static::$entityTypeId === 'user') {
    +      $this->assertResourceErrorResponse(405, 'No route found for "POST ' . str_replace($this->baseUrl, '', $this->getPostUrl()->setAbsolute()->toString()) . '": Method Not Allowed (Allow: GET, HEAD)', $response);
    +    }
    +    else {
    +      $this->assertResourceErrorResponse(404, 'No route found for "POST ' . str_replace($this->baseUrl, '', $this->getPostUrl()->setAbsolute()->toString()) . '"', $response);
    +    }
    

    We need a documentation for these things

  3. +++ b/core/modules/user/user.routing.yml
    @@ -117,6 +117,7 @@ user.page:
         _controller: '\Drupal\user\Controller\UserController::userPage'
         _title: 'My account'
       requirements:
    +    _method: 'GET'
         _user_is_logged_in: 'TRUE'
     
    

    I think this solution makes sense. In general I'm wondering whether it might be a problem on way more routes conceptually.

Wim Leers’s picture

Title: EntityResource POST routes all use the confusing default in <8.3: use entity types' https://www.drupal.org/link-relations/create link template if available » [PP-1] EntityResource POST routes all use the confusing default in <8.3: use entity types' https://www.drupal.org/link-relations/create link template if available
Status: Needs work » Postponed
  1. Agreed!
  2. Yes, it can be a problem for more routes conceptually: any time a route uses the same path as a non-HTML route.

#157 failed with 2 fails, but they're now in PathValidatorTest. Because I failed to update that test; it needs additional services to be injected. Rather than fixing that, I'm now postponing this issue on #2822190: PathValidator validates based on a RequestContext leaked from the current request, resulting in false negatives during CLI requests and POST submissions. Once that's in, #157's interdiff can be reverted and this patch will be green!

Wim Leers’s picture

Status: Postponed » Needs review
FileSize
3.86 KB
30.29 KB
Wim Leers’s picture

Status: Needs review » Postponed
Wim Leers’s picture

Status: Postponed » Needs review
FileSize
26.24 KB

#2815845: Importing (deploying) REST resource config entities should automatically do the necessary route rebuilding conflicted with this. Rerolling this patch. #152 is still the actual latest patch. #165 is just #152 + #2822190-40: PathValidator validates based on a RequestContext leaked from the current request, resulting in false negatives during CLI requests and POST submissions.

So, rerolling #152. This should fail again with the same failures as #152: those in ContactSitewideTest and ContactStorageTest. No interdiff because rebase, with the only conflict (in rest.services.yml) resolved.

Wim Leers’s picture

Title: [PP-1] EntityResource POST routes all use the confusing default in <8.3: use entity types' https://www.drupal.org/link-relations/create link template if available » EntityResource POST routes all use the confusing default in <8.3: use entity types' https://www.drupal.org/link-relations/create link template if available
FileSize
1.15 KB
27.35 KB

An alternative could be to not block this on #2822190: PathValidator validates based on a RequestContext leaked from the current request, resulting in false negatives during CLI requests and POST submissions and just modify ContactSitewideTest. This is the line that's causing a failure:

    $this->updateContactForm($id, $label = $this->randomMachineName(16), $recipients_str = implode(',', array($recipients[0], $recipients[1])), $reply = $this->randomMachineName(30), FALSE, 'Your message has been sent.', '/user');

Specifically, the /user parameter at the end. That's filling out a "contact form" config entity configuration form to set the redirect path to be /user. This then causes problems because:

  1. This patch modifies the /user route to only allow the GET method to be used
  2. And the PathValidator has an important bug, described & being fixed in #2822190
  3. Plus ContactSitewideTest just happens to have chosen this particular URL!

What an unfortunate coincidence.

So why did that test use that particular path? Well, turns out that is coincidence too: @naveenvalecha wrote this at #306662-135: Add redirect option to site-wide contact forms:

1) I see no /user setting why test using that?

The default front page is /user b/c we are not installing the system_test module and not manually setting the front page.

i.e. for no other reason that that is the default front page, and is guaranteed to be available to redirect to, and there are no other non-admin URLs installed in this particular test. We could install the system_test module and just point somewhere else.

But why even set this redirect path? Because we're testing the redirect functionality of the Contact module, DUH! Well… not setting any redirect path actually is also okay, because we're explicitly testing the redirect functionality separately anyway — see this further down in ContactSitewideTest:

    // Test messages and redirect.
    /** @var \Drupal\contact\ContactFormInterface $form */
    $form = ContactForm::load($contact_form);
    $form->setMessage('Thanks for your submission.');
    $form->setRedirectPath('/user/' . $admin_user->id());
    $form->save();
    // Check that the field is displayed.
    $this->drupalGet('contact/' . $contact_form);

    // Submit the contact form and verify the content.
    $edit = array(
      'subject[0][value]' => $this->randomMachineName(),
      'message[0][value]' => $this->randomMachineName(),
      $field_name . '[0][value]' => $this->randomMachineName(),
    );
    $this->drupalPostForm(NULL, $edit, t('Send message'));
    $this->assertText('Thanks for your submission.');
    $this->assertUrl('user/' . $admin_user->id());

Conclusion: we can safely change that '/user' to FALSE, and tests pass. There was no good reason for us to pick that particular path. And in fact, we don't even need that at all, and we still don't lose any test coverage.

Is this incredibly silly, to the point of being hilarious? YES.

The last submitted patch, 167: 2293697-167.patch, failed testing.

tedbow’s picture

Status: Needs review » Needs work
+++ b/core/modules/contact/src/Tests/ContactSitewideTest.php
@@ -156,7 +156,7 @@ function testSiteWideContact() {
-    $this->updateContactForm($id, $label = $this->randomMachineName(16), $recipients_str = implode(',', array($recipients[0], $recipients[1])), $reply = $this->randomMachineName(30), FALSE, 'Your message has been sent.', '/user');
+    $this->updateContactForm($id, $label = $this->randomMachineName(16), $recipients_str = implode(',', array($recipients[0], $recipients[1])), $reply = $this->randomMachineName(30), FALSE, 'Your message has been sent.', FALSE);

I think the this update makes sense because as @Wim Leers said we are testing the redirect functionality elsewhere in this test.

+++ b/core/modules/rest/tests/src/Functional/EntityResource/EntityResourceTestBase.php
@@ -710,6 +715,17 @@ public function testPost() {
+
+    // BC: old default POST URLs have their path updated by the inbound path
+    // processor \Drupal\rest\PathProcessor\PathProcessorEntityResourceBC to the
+    // new URL, which is derived from the 'create' link template if an entity
+    // type specifies it.
+    if ($this->entity->getEntityType()->hasLinkTemplate('create')) {
+      $this->entityStorage->load(static::$secondCreatedEntityId)->delete();
+      $old_url = Url::fromUri('base:entity/' . static::$entityTypeId);
+      $response = $this->request('POST', $old_url, $request_options);
+      $this->assertResourceResponse(201, FALSE, $response);
+    }

This looks good for testing core entities because obviously client code will already be using /entity/[entity_type] path and we should make sure it is still working.

The only thing I am wondering about is:
Since all core entity types now do have a "create" link template we are no longer testing that an entity type does not have this. Obviously there will be custom and contrib entity types that do not have "create" link.

So we should test that this still works.

Chatted with @Wim Leers about this and he suggested removing
"create" = "/entity_test",
from \Drupal\entity_test\Entity\EntityTest to prove that it still works if "create" is not there.

That sounds like a good idea to me. Also we should probably comment in EntityTest that "create" is intentionally omitted so we don't accidentally add it back thinking it was not on purpose.

Wim Leers’s picture

Status: Needs work » Needs review
FileSize
1007 bytes
27.36 KB

Great find :)

So, removed the create link template from EntityTest's annotation. Left a comment.

This then still passes tests because the test coverage was already written with entity types that don't specify create link templates in mind:

+    // BC: old default POST URLs have their path updated by the inbound path
+    // processor \Drupal\rest\PathProcessor\PathProcessorEntityResourceBC to the
+    // new URL, which is derived from the 'create' link template if an entity
+    // type specifies it.
+    if ($this->entity->getEntityType()->hasLinkTemplate('create')) {
+      $this->entityStorage->load(static::$secondCreatedEntityId)->delete();
+      $old_url = Url::fromUri('base:entity/' . static::$entityTypeId);
+      $response = $this->request('POST', $old_url, $request_options);
+      $this->assertResourceResponse(201, FALSE, $response);
+    }
Wim Leers’s picture

FileSize
1.42 KB
27.46 KB

I hadn't fixed #162.2 yet. Done.

effulgentsia’s picture

Status: Needs review » Postponed
+++ b/core/modules/contact/src/Tests/ContactSitewideTest.php
@@ -156,7 +156,7 @@ function testSiteWideContact() {
-    $this->updateContactForm($id, $label = $this->randomMachineName(16), $recipients_str = implode(',', array($recipients[0], $recipients[1])), $reply = $this->randomMachineName(30), FALSE, 'Your message has been sent.', '/user');
+    $this->updateContactForm($id, $label = $this->randomMachineName(16), $recipients_str = implode(',', array($recipients[0], $recipients[1])), $reply = $this->randomMachineName(30), FALSE, 'Your message has been sent.', FALSE);

+1, per #168. I'm glad that makes this issue's tests pass.

+++ b/core/modules/user/user.routing.yml
@@ -117,6 +117,7 @@ user.page:
   requirements:
+    _method: 'GET'

I don't think we can commit this line to core prior to #2822190: PathValidator validates based on a RequestContext leaked from the current request, resulting in false negatives during CLI requests and POST submissions being fixed. Even though I just bumped that issue to Critical on its own, committing this change would enlarge that problem, because anywhere that a '/user' path is being validated (contact form redirect, menu link form, path alias form, ...) would stop working. That's already a problem for other routes with _method requirements, but we shouldn't expand the breakage to more routes until we fix that issue.

Marking this postponed to reflect this. But if folks here want to reset to Needs review in order to keep reviewing the rest of the patch, that's fine.

effulgentsia’s picture

Status: Postponed » Needs review
+++ b/core/modules/user/src/Entity/User.php
@@ -51,6 +51,7 @@
+ *     "create" = "/user",

Um, what if we just remove this (and the second hunk of #173) from this patch and punt it to a follow-up that's postponed on #2822190: PathValidator validates based on a RequestContext leaked from the current request, resulting in false negatives during CLI requests and POST submissions? But then the rest of this can proceed without waiting for that?

Wim Leers’s picture

That is correct! Let's do that. Great alternative. :)

effulgentsia’s picture

Status: Needs review » Needs work

NW per #174/#175.

Wim Leers’s picture

tedbow’s picture

Status: Needs review » Reviewed & tested by the community

#177 looks good. Just #172 with adding "create" link template for user entity and removing necessary test changes. Follow up and change record look good.

RTBC!

alexpott’s picture

@effulgentsia said he might look at this. I've been staring at this issue and have a couple of thoughts. I'm not sure if they block or commit or not and I'm happy to defer to @effulgentsia's judgement if he decides to commit this.

  1. +++ b/core/core.services.yml
    @@ -1143,7 +1143,7 @@ services:
         class: Drupal\Core\Access\CsrfRequestHeaderAccessCheck
         arguments: ['@session_configuration', '@csrf_token']
         tags:
    -      - { name: access_check }
    +      - { name: access_check, needs_incoming_request: TRUE }
    

    Ideally this would be in a separate issue as it is not related to the issue being fixed.

  2. +++ b/core/modules/node/src/Controller/NodeViewController.php
    @@ -56,6 +57,17 @@ public function view(EntityInterface $node, $view_mode = 'full', $langcode = NUL
         foreach ($node->uriRelationships() as $rel) {
           $url = $node->toUrl($rel);
    +
    +      // It's not guaranteed that every link relation type also has a
    +      // corresponding route. For some, additional modules or configuration may
    +      // be necessary.
    +      try {
    +        $generated_url = $url->toString();
    +      }
    +      catch (RouteNotFoundException $e) {
    +        continue;
    +      }
    

    This chance is a shame. It means that this is likely to require fixes to contrib that have similar code for.

Wim Leers’s picture

#179

  1. I agree that ideally this is the case. Sadly, it's also blocking this, and it was only discovered as part of this issue.
  2. True. There is one other possible approach: change \Drupal\Core\Entity\Entity::uriRelationships() from
      public function uriRelationships() {
        return array_keys($this->linkTemplates());
      }
    

    to

      public function uriRelationships() {
        return array_filter(array_keys($this->linkTemplates()), function ($link_relation_type) {
          $url_exists = …;
          return $url_exists ? TRUE : FALSE;
        });
      }
    

    i.e. update its logic to only return URI relationships for which there actually is a URI. That is actually even consistent with its documentation:

      /**
       * Gets a list of URI relationships supported by this entity.
       *
       * @return string[]
       *   An array of link relationships supported by this entity.
       */
      public function uriRelationships();
    

    It's then still possible to get all link relation types (and link templates) for an entity type, by calling EntityType::getLinkTemplates().

Thoughts?

effulgentsia’s picture

Status: Reviewed & tested by the community » Needs work
  1. +++ b/core/modules/node/src/Entity/Node.php
    @@ -71,6 +71,7 @@
    + *     "create" = "/node",
    

    Kind of weird to be defining a link template that doesn't have a route unless you enable other modules. Even weirder is that a non-REST route does exist at this path if you have the Views module and the Frontpage View enabled. And weirdest of all, that route has nothing to do with "creating".

  2. +++ b/core/modules/rest/src/EventSubscriber/EntityResourcePostRouteSubscriber.php
    @@ -0,0 +1,74 @@
    +        $entity_create_route_name = "entity.$entity_type_id.create";
    +        $route_collection->add($entity_create_route_name, $rest_post_route);
    

    So, if I enable rest, hal, and basic_auth modules, then this code runs for the node entity type, due to rest/config/optional/rest.resource.entity.node.yml. But if I then disable one of those modules, e.g., hal, this code doesn't run. This has an interesting side-effect: if I disable the Frontpage View, and enable rest, hal, and basic_auth modules, then when I navigate to the frontpage or '/node', I get the "The website encountered an unexpected error. Please try again later." WSOD. If I then disable HAL module, I stop getting that error. So, setting this issue to NW at a minimum for that.

  3. +++ b/core/modules/node/src/Controller/NodeViewController.php
    @@ -56,6 +57,17 @@ public function view(EntityInterface $node, $view_mode = 'full', $langcode = NUL
    +      // It's not guaranteed that every link relation type also has a
    +      // corresponding route. For some, additional modules or configuration may
    +      // be necessary.
    +      try {
    +        $generated_url = $url->toString();
    +      }
    +      catch (RouteNotFoundException $e) {
    +        continue;
    +      }
    

    #180.2 sounds appealing, but let's discuss it in a separate issue, since it has BC implications. But even if we do that, this part of the code is still weird, because in the case that there is a entity.node.create route, we end up outputting <link rel="create" href="/node?node=1" /> to the HTML view of /node/1. But that makes no sense, since per #1, with the regular HTML browser, I can't actually create a node at that URL.

Wim Leers’s picture

  1. Why is this strange? The entity type is defining what link template to use for certain functionality. That doesn't need to imply that that functionality is also provided.

    Why is it strange that something else may use that same path? It's a key principle of the web that you can do

    GET /resource HTTP/1.1
    Accept: text/html
    

    and

    GET /resource HTTP/1.1
    Accept: application/json
    

    and get completely different responses. The same URL can serve different representations. This is exactly like that. Your example is exactly like that.

    If you follow your argument, then the logical conclusion is that we should have used a path prefix for every REST URL.

  2. All of this is entirely due to limitations/flaws in the REST module's architecture as well as Symfony's routing system. See #127 for a detailed explanation.
  3. Well, not doing #180.2 also has BC implications.

    I don't see what the problem is with <link rel="https://www.drupal.org/link-relations/create" href="/node" /> being output? Link relation headers are for machines. It's up to machines to interpret/use those headers. Not all of these links are necessarily intended for HTML use only.

    E.g. the dns-prefetch link relation type is entirely unrelated to HTML. So are hosts, lrdd and others. These are all official ones, see https://www.iana.org/assignments/link-relations/link-relations.xhtml.

Wim Leers’s picture

Status: Needs work » Reviewed & tested by the community

So, #181 was marking NW for something that doesn't need work IMHO.

But #181.3 (regarding #180.2) about NodeViewController is something that would need to happen, but apparently @effulgentsia believes this can happen in a follow-up. And apparently the logic for generating link headers in NodeViewController is broken anyway, because we indeed get <link rel="create" href="/node?node=1" />, whereas we should be getting <link rel="https://www.drupal.org/link-relations/create" href="/node" />. So we need to fix NodeViewController anyway; it's already broken in HEAD.
Therefore I'm fine with postponing #180.2/#181.3 to a follow-up.

That then means this is still RTBC IMHO.

alexpott’s picture

I think #180.2 looks like a good idea and will mean that contrib code doesn't need to change because of this change. I think the BC implications of not making that change are way worse than doing it. If we don't do it, then code that used to not throw an exception now will.

Wim Leers’s picture

Does 180.2 block commit or can it be a major follow-up that lands in a few days?

xjm’s picture

Does 180.2 block commit or can it be a major follow-up that lands in a few days?

We should never assume that any followup will ever land. :) We should only put things in followups if we are willing to accept them never landing.

Personally I think a followup sounds like a good idea if needed for any of those points in #180; let's check with effulgentsia though before making a commit(ment) here.

alexpott’s picture

For me #180.2 blocks commit - but @effulgentsia is not awake yet - going to discuss with him once he is up.

Wim Leers’s picture

Implemented #180.2.

Wim Leers’s picture

For me #180.2 blocks commit - but @effulgentsia is not awake yet - going to discuss with him once he is up.

I personally agree with that.

Wim Leers’s picture

Bonus:

  • #177: 19 files changed, 236 insertions(+), 18 deletions(-)
  • #188: 18 files changed, 211 insertions(+), 15 deletions(-)

Fewer changes :)

alexpott’s picture

Thanks @Wim Leers I think the interdiff shows why #180.2 was good to fix.

Also as pointed out by @Wim Leers the docs for uriRelationships() says that it returns supported relationships. Returning a relationship that can cause RouteNotFoundException would be against that.

  /**
   * Gets a list of URI relationships supported by this entity.
   *
   * @return string[]
   *   An array of link relationships supported by this entity.
   */
  public function uriRelationships();
effulgentsia’s picture

Status: Reviewed & tested by the community » Needs review
+++ b/core/lib/Drupal/Core/Entity/Entity.php
@@ -323,7 +324,19 @@ protected function urlRouteParameters($rel) {
   public function uriRelationships() {
-    return array_keys($this->linkTemplates());
+    return array_filter(array_keys($this->linkTemplates()), function ($link_relation_type) {
+      // It's not guaranteed that every link relation type also has a
+      // corresponding route. For some, additional modules or configuration may
+      // be necessary. The interface demands that we only return supported URI
+      // relationships.
+      try {
+        $this->toUrl($link_relation_type)->toString(TRUE)->getGeneratedUrl();
+      }
+      catch (RouteNotFoundException $e) {
+        return FALSE;
+      }
+      return TRUE;
+    });

+1. This is great.

All of this is entirely due to limitations/flaws in the REST module's architecture as well as Symfony's routing system. See #127 for a detailed explanation.

Regardless of where the flaw lies, I don't think the patch is committable if by encountering the flaw, it's creating a WSOD. Clarifying #181.2. If I do this in HEAD:

  1. Install Standard
  2. Enable rest, hal, and basic_auth modules.
  3. Disable the Frontpage View.
  4. Visit either / or /node.

Then in visiting either of those last 2 URLs, I get a nicely themed "Page not found" page, which is correct.

If I then apply #188, then visiting either of those URLs shows me a white screen of death, with the infamous "The website encountered an unexpected error. Please try again later." message.

The above problem on its own would be at minimum a Major bug ("Trigger a PHP error through the user interface, but only under rare circumstances") if you consider disabling the Frontpage View to be a rare circumstance. Except this can also be triggered by disabling the Views module itself too. Furthermore, you can fix the error at / by editing the frontpage path at /admin/config/system/site-information, but the WSOD still remains when you visit /node. And I think that's made even worse by the <link rel="create" href="/node?node=1" /> being output from /node/1, since then you're telling the user agent that there's a resource at /node but there isn't (at least not one available to this user agent).

Setting to Needs review for more discussion on whether others agree or disagree with me about this being a commit blocker.

dawehner’s picture

if you consider disabling the Frontpage View to be a rare circumstance.

To be honest though its kind of good pratice to disable the frontpage view. I'm looking into that for a while.

dawehner’s picture

To solve this properly we IMHO should catch any client side errors.

After applying the latest changes you get a

A client error happened

page, not rendered as nicely as the 403/404 pages, but at least something.

The last submitted patch, 194: 2293697-194.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 194: interdiff-2293697-194.patch, failed testing.

alexpott’s picture

+++ b/core/lib/Drupal/Core/EventSubscriber/DefaultExceptionHtmlSubscriber.php
@@ -80,6 +80,18 @@ protected function getHandledFormats() {
+  public function on4xx(GetResponseForExceptionEvent $event) {

If we do add this to solve the problem described in #192 we need to test it.

xjm’s picture

Title: EntityResource POST routes all use the confusing default in <8.3: use entity types' https://www.drupal.org/link-relations/create link template if available » EntityResource POST routes all use the confusing default: use entity types' https://www.drupal.org/link-relations/create link template if available
Wim Leers’s picture

The website encountered an unexpected error. Please try again later.
That's what you would get when you have verbose errors disabled. But people developing sites don't have that disabled; they have it enabled. Then they get this instead:

The website encountered an unexpected error. Please try again later.

Symfony\Component\HttpKernel\Exception\MethodNotAllowedHttpException: No route found for "GET /node": Method Not Allowed (Allow: POST) in Symfony\Component\HttpKernel\EventListener\RouterListener->onKernelRequest() (line 180 of vendor/symfony/http-kernel/EventListener/RouterListener.php).
Drupal\Core\Routing\LazyRouteFilter->filter(Object, Object) (Line: 247)
Drupal\Core\Routing\Router->applyRouteFilters(Object, Object) (Line: 151)
Drupal\Core\Routing\Router->matchRequest(Object) (Line: 90)
Drupal\Core\Routing\AccessAwareRouter->matchRequest(Object) (Line: 154)
Symfony\Component\HttpKernel\EventListener\RouterListener->onKernelRequest(Object, 'kernel.request', Object) (Line: 111)
Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher->dispatch('kernel.request', Object) (Line: 120)
Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object, 1) (Line: 62)
Symfony\Component\HttpKernel\HttpKernel->handle(Object, 1, 1) (Line: 57)
Drupal\Core\StackMiddleware\Session->handle(Object, 1, 1) (Line: 47)
Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object, 1, 1) (Line: 99)
Drupal\page_cache\StackMiddleware\PageCache->pass(Object, 1, 1) (Line: 78)
Drupal\page_cache\StackMiddleware\PageCache->handle(Object, 1, 1) (Line: 47)
Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object, 1, 1) (Line: 50)
Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object, 1, 1) (Line: 23)
Stack\StackedHttpKernel->handle(Object, 1, 1) (Line: 656)
Drupal\Core\DrupalKernel->handle(Object) (Line: 19)

That's actually a very helpful error!


You can easily also get exactly the same at every https://www.drupal.org/link-relations/create link relation URL. For example, for taxonomy terms:

uuid: a323310e-66cb-48ab-b3d2-006b63b9fc15
langcode: en
status: true
dependencies:
  module:
    - basic_auth
    - hal
    - taxonomy
id: entity.taxonomy_term
plugin_id: 'entity:taxonomy_term'
granularity: method
configuration:
  POST:
    supported_formats:
      - hal_json
    supported_auth:
      - basic_auth

Now GET /taxonomy/term results in the exact same error as the one you reported for /node.


The reason for those error responses is that you're accessing this URL without specifying a format (i.e. without ?_format=…). Therefore the default of HTML is assumed. And the default exception subscriber for HTML only handles 401, 403 and 404.
I'm not sure #194 is the best approach. It's technically correct ("handle all 4xx errors other than 401/403/404"), but does it also match the expected behavior? Wouldn't it from a HTML/browser POV make more sense to map e.g. 405 errors (which is what we're seeing here) to a 404, because… there is no HTML to be found? Which is semantically equivalent with a 404. EDIT: that also happens to be the behavior before this patch, but that's not even why I'm proposing this.

I disagree that improving the handling of accessing a "REST-only route" without specifying a format is something we need to improve here. It's a pre-existing problem. It's a problem that's out of scope for this issue. And I think the scenario in #192 is fairly contrived. I don't see what the big problem is with making this a bit more likely to occur. It's a problem we need to solve regardless of this issue.

(And this once again goes to show that Drupal is still far away from being "API-first".)

dawehner’s picture

Which is semantically equivalent with a 404

This is weird. This kinda of mixes up different layers. Why should HTML care about the status code and wise versa.
It feels like we are so used to the idea of 403/404 as users, that we somehow think that they are special.

effulgentsia’s picture

And apparently the logic for generating link headers in NodeViewController is broken anyway, because we indeed get <link rel="create" href="/node?node=1" />, whereas we should be getting <link rel="https://www.drupal.org/link-relations/create" href="/node" />. So we need to fix NodeViewController anyway; it's already broken in HEAD.

True, this is a pre-existing problem. Even without this patch's addition of the "create" relation, we're already encountering this problem with the "delete-form" relation. The closest issue we have currently open for fixing it is #2848945: EntityType link templates: using link relation type names as keys is a problem. This was actually leading to HTML validation problems, which we provided a partial workaround for in #2406533: edit-form, delete-form etc. <link> tags added on /node/{node} are invalid according to W3C Validator. Except that workaround only sidestepped it for anonymous users, which means we're still producing invalid HTML for authenticated users. Plus, that workaround was only applied to nodes, which means we're still producing invalid HTML for authenticated and anonymous users for taxonomy terms: #2821635: edit-form, delete-form etc. <link> tags added on /taxonomy/term/{taxonomy_term} are invalid according to W3C Validator.

The issue summary of #2406533: edit-form, delete-form etc. <link> tags added on /node/{node} are invalid according to W3C Validator says:

What is certain is that having these links present on page for anonymous users is affecting how Google and other service engines index sites. Google will visit each link relation and get many access denieds.

Do we know for sure that Google and other service engines access all link relations? Even ones not registered with IANA, such as "delete-form" and "create"? In which case, this patch is adding another one ("create") for it to try to access and fail. Maybe that's ok? It's just one more 4xx response for the engine to deal with. Except, while in HEAD, we have pre-existing ones that return 403s, this patch adds one that returns a 405. Again, maybe that's ok, but might be good to do some research on whether it is or not. #2406533-113: edit-form, delete-form etc. <link> tags added on /node/{node} are invalid according to W3C Validator said that even the 403s for the relation links coming from node pages was a severe bug. I don't know if adding links to 405 pages for authenticated node traffic or anonymous taxonomy term traffic would add additional substantial severity to the remaining existing problems in HEAD.

Wim Leers’s picture

Do we know people at Google who can answer this with confidence? The fact that it's so hard to find information about this when searching the web means we're probably the first ones to do this, or the first major CMS at least.

xjm’s picture

Keep in mind that there's a reasonable chance that someone at Google couldn't answer the question because I'd bet it might be covered by NDA. (Since gaming search engine rankings is something they have to be continually watchful about, and etc.)

Wim Leers’s picture

Another two weeks that have passed without progress. We're effectively blocking a massive DX improvement/the removal of a major Drupal WTF from our REST API on SEO matters. That's unfortunate :(

I also pinged somebody at Google, hopefully he'll point us in the right direction: https://twitter.com/wimleers/status/837293938084040704.

Wim Leers’s picture

Status: Needs work » Needs review

I got a response from the person at Google:

That's no conclusive evidence, but a strong indicator that Google indeed does not penalize this.

In the mean time I've found more evidence that link relations (<link rel=""> or Link header) indeed do not need to be tied to a HTML presentation:

  1. <link rel="search" type="application/opensearchdescription+xml" href="https://s1.wp.com/opensearch.xml" title="WordPress.com" /> — this is clearly serving an XML file that is completely unusable for a human user, but is relevant to a particular type of user agent: one that renders HTML.
  2. <link rel='dns-prefetch' href='//s2.wp.com' /> — this is not serving anything, and is a directive for low-level implementation details of HTML-rendering browsers, to accelerate the fetching of referenced resources (CSS, JS, images)
  3. <link rel="pingback" …> — which was explicitly <a href="https://github.com/whatwg/html/commit/d5ff270eef4e4087a3c3652bb8a0a5f53345361e">incorporated into the HTML 5 spec</a>. As per the <a href="http://hixie.ch/specs/pingback/pingback-1.0#TOC1.1">Pingback spec</a>, this will cause an XML-RPC call to the referenced site, to notify (ping) it. Let me quote the one sentence that captures it all:
    <blockquote>
    The pingback mechanism uses an HTTP header and an HTML or XHTML <code><link>

    element for autodiscovery, and uses a single XML-RPC call for notifying the target site of the link on the source site.

    This is NOT something that a browsing user agent even can do. It's information specifically for other sites. Not a robot or human browsing the internet. It's for a site to act as a user agent whenever content is posted that references content hosted on a pingback-enabled site.

  4. The describes link relation type specifically allows one resource accessible via HTTP to express that it describes the structure of another resource. This is RFC6892. It mentions RDF, JSON and XML. And it has this gem:

    However, since link relations are independent of resource formats or representations, such an association could also be made in other formats such as XML or JavaScript Object Notation (JSON), allowing servers to use a single and consistent link relation to associate description resources with described resources.

    (Emphasis mine.)

In other words: link relations are not meant to be HTML-specific or HTML-centric. They are specifically meant to be representation-agnostic. It's up to the user agent to first inspect the rel, and check if this particular link relation type is something that this user agent A) supports, B) surfaces to the end user in some way.

I also find this confirmed in the very introduction of the RFC that standardized the concept of link relation types, RFC5988:

A means of indicating the relationships between resources on the Web, as well as indicating the type of those relationships, has been available for some time in HTML, and more recently in Atom. These mechanisms, although conceptually similar, are separately specified. However, links between resources need not be format specific; it can be useful to have typed links that are independent of their serialisation, especially when a resource has representations in multiple formats.

To this end, this document defines a framework for typed links that isn't specific to a particular serialisation or application. It does so by redefining the link relation registry established by Atom to have a broader domain, and adding to it the relations that are defined by HTML.

And from section 4:

In the simplest case, a link relation type identifies the semantics of a link. For example, a link with the relation type "copyright" indicates that the resource identified by the target IRI is a statement of the copyright terms applying to the current context IRI.

Link relation types can also be used to indicate that the target resource has particular attributes, or exhibits particular behaviours; for example, a "service" link implies that the identified resource is part of a defined protocol (in this case, a service description).

Relation types are not to be confused with media types; they do not identify the format of the representation that results when the link is dereferenced. Rather, they only describe how the current context is related to another resource.

Wim Leers’s picture

Assigned: Wim Leers » Unassigned
FileSize
3.28 KB
25.62 KB

I believe the above presents overwhelming evidence that the approach in the current patch is justified. And that we do not need to distinguish between "HTML link relation types" and "non-HTML link relation types".

Hence we wouldn't need to include @dawehner's proposed changes in #194.

But @dawehner's changes still help the edge case that @effulgentsia described in #192: of removing the front page view and enabling the REST+HAL modules, then getting plain text 405 responses for /node + /.

So, let's keep the changes made by @dawehner in #194, but make them pass tests.


#2248983: Define the revision metadata base fields in the entity annotation in order for the storage to create them only in the revision table conflicted with this slightly. Rebased too.

jofitz’s picture

Status: Needs review » Needs work
Issue tags: +Needs reroll
jofitz’s picture

Status: Needs work » Needs review
Issue tags: -Needs reroll
FileSize
24.35 KB

Re-rolled, ensuring [] array syntax.

Wim Leers’s picture

Manually diffed #206 and #208. Confirming this is a correct straight rebase. (The size difference is attributable to the fact that #206 is rolled with diff stats enabled, and #208 is not.)

Munavijayalakshmi’s picture

Assigned: Unassigned » Munavijayalakshmi
Status: Needs review » Needs work
  1. +++ b/core/modules/rest/src/EventSubscriber/EntityResourcePostRouteSubscriber.php
    @@ -0,0 +1,74 @@
    +        // Create a route for the 'create' link relation type for this entity type
    

    Line exceeding 80 characters.

  2. +++ b/core/modules/rest/src/EventSubscriber/EntityResourcePostRouteSubscriber.php
    @@ -0,0 +1,74 @@
    +        // that uses the same route definition as the REST 'POST' route which use
    

    Line exceeding 80 characters.

  3. +++ b/core/modules/rest/src/PathProcessor/PathProcessorEntityResourceBC.php
    @@ -0,0 +1,55 @@
    +      // specific one if it exists)
    

    Comments should (noramlly) begin with a capital letter and end with a full stop / period .

Munavijayalakshmi’s picture

Assigned: Munavijayalakshmi » Unassigned
Status: Needs work » Needs review
FileSize
24.35 KB
1.9 KB
dawehner’s picture

  1. +++ b/core/lib/Drupal/Core/Entity/Entity.php
    @@ -323,7 +324,19 @@ protected function urlRouteParameters($rel) {
    +      try {
    +        $this->toUrl($link_relation_type)->toString(TRUE)->getGeneratedUrl();
    +      }
    +      catch (RouteNotFoundException $e) {
    

    The only weird bit about this code is that we now generate each URL twice. Once here, and once when we actually render the link relationship. Is this something we should worry about?

  2. +++ b/core/modules/rest/src/EventSubscriber/EntityResourcePostRouteSubscriber.php
    @@ -0,0 +1,74 @@
    +      $entity_type_id = substr($plugin_id, 7);
    +      $rest_post_route_name = "rest.entity.$entity_type_id.POST";
    +      if ($rest_post_route = $route_collection->get($rest_post_route_name)) {
    +        // Create a route for the 'create' link relation type for this entity
    ...
    +        $route_collection->add($entity_create_route_name, $rest_post_route);
    

    I'm wondering whether we should add the route inside ResourceBase

Wim Leers’s picture

#212:

  1. Good point, that's true! We could use RouteProvider::getRouteByName()… but it'd be rather difficult to actually do that, because we also allow an entity type to specify an uri_callback, which is what is used if a particular link template is not specified:
      if (isset($link_templates[$rel])) {
        // route-based logic
      }
      else {
          …
    
          // Invoke the callback to get the URI. If there is no callback, use the
          // default URI format.
          if (isset($uri_callback) && is_callable($uri_callback)) {
            $uri = call_user_func($uri_callback, $this);
          }
          else {
            throw new UndefinedLinkTemplateException("No link template '$rel' found for the '{$this->getEntityTypeId()}' entity type");
          }
      }
    

    The support for uri_callback means we cannot simply rely on RouteProvider::getRouteByName(). Which means we need to call the URI callback, which means we need to actually generate the URL after all AFAICT :(

  2. I tried that (in #127). It's impossible. Because whatever ResourceBase (or any \Drupal\rest\Plugin\ResourceInterface) implementation returns in its routes() method, the route names that that returns are always prefixed by rest. in \Drupal\rest\Routing\ResourceRoutes. Let me quote what I wrote in #127:

    Letting \Drupal\rest\Plugin\rest\resource\EntityResource::routes() generate a entity.node.create route with such a duplicate definition doesn't work, because \Drupal\rest\Routing\ResourceRoutes::alterRoutes() prefixes all routes with 'rest'.
    I can make that work by also modifying ResourceRoutes::alterRoutes() to no longer set the rest. prefix on route names, and just setting that prefix in EntityResource and DBLogResource. But this would break BC for other REST resources — their route names would change.

dawehner’s picture

The support for uri_callback means we cannot simply rely on RouteProvider::getRouteByName(). Which means we need to call the URI callback, which means we need to actually generate the URL after all AFAICT :(

At least for me the uri_callback concept is just fundamentally flawed and I would be surprised it actually works properly in all cases still.

I tried that (in #127). It's impossible. Because whatever ResourceBase (or any \Drupal\rest\Plugin\ResourceInterface) implementation returns in its routes() method, the route names that that returns are always prefixed by rest. in \Drupal\rest\Routing\ResourceRoutes. Let me quote what I wrote in #127:

Ah, why can't everything be nice. I wonder whether you could still generate the routes there, but then in EntityResourcePostRouteSubscriber just renames the routes?

Wim Leers’s picture

Well, uri_callback is still an official API, and supported, so we can't really ignore that I think?

Regarding generating routes in ResourceBase and then renaming them in the event subscriber: that'd be possible too, yes. I'm not sure whether it's better. Both approaches are very similar. I don't have a strong opinion on either approach. Do you want me to change it to what you proposed?

dawehner’s picture

Regarding generating routes in ResourceBase and then renaming them in the event subscriber: that'd be possible too, yes. I'm not sure whether it's better. Both approaches are very similar. I don't have a strong opinion on either approach. Do you want me to change it to what you proposed?

It would be nice if plugins could easily change the route somehow, if needed.

Wim Leers’s picture

But when would that ever be needed? The whole point is that this create route is just pointing to the exact same route definition. You can change the route definition to anything you want in your ResourceInterface-implementing plugin. The entity.$entity_type_id.create that we create automatically would just use exactly that same route definition.

That's all this is: two route names using the same route definition, so that you can generate URLs to this route using either route name.

dawehner’s picture

Status: Needs review » Reviewed & tested by the community

Oh sorry, I totally forgot what this patch was doing, aka. exactly what I though should be done :)

Wim Leers’s picture

Hah! Great :D

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 211: 2293697-211.patch, failed testing.

jofitz’s picture

Status: Needs work » Needs review
FileSize
23.86 KB

Patch no longer applies - re-rolled.

Status: Needs review » Needs work

The last submitted patch, 221: 2293697-221.patch, failed testing.

jofitz’s picture

Status: Needs work » Needs review
FileSize
864 bytes
23.9 KB

Sorry, messed up that re-roll. Let's try again.

Wim Leers’s picture

Status: Needs review » Reviewed & tested by the community

Conflicted in EntityResourceTestBase due to changes introduced in #2853211: EntityResource::post() incorrectly assumes that every entity type has a canonical URL.

Back to RTBC!

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 223: 2293697-223.patch, failed testing.

dawehner’s picture

Status: Needs work » Reviewed & tested by the community

That seemed to be a random fluctuation.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 223: 2293697-223.patch, failed testing.

Wim Leers’s picture

Status: Needs work » Reviewed & tested by the community

Again.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 223: 2293697-223.patch, failed testing.

dawehner’s picture

Status: Needs work » Needs review
FileSize
24.09 KB

Just a quick reroll

Wim Leers’s picture

Status: Needs review » Reviewed & tested by the community

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 230: 2293697-230.patch, failed testing.

Wim Leers’s picture

Status: Needs work » Reviewed & tested by the community
FileSize
24.04 KB

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 233: 2293697-233.patch, failed testing.

Wim Leers’s picture

Status: Needs work » Reviewed & tested by the community

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 233: 2293697-233.patch, failed testing.

dawehner’s picture

Status: Needs work » Reviewed & tested by the community

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 233: 2293697-233.patch, failed testing.

Wim Leers’s picture

Status: Needs work » Reviewed & tested by the community
FileSize
24.07 KB

This conflicted with #2869415: EntityResourceTestBase::getUrl() overrides BrowserTestBase::getUrl(), yet offers different functionality, but the conflict was trivial to resolve :) (EntityResourceTestBase::getPostUrl()
was renamed to EntityResourceTestBase::getEntityResourcePostUrl().)

alexpott’s picture

Status: Reviewed & tested by the community » Fixed
Issue tags: +8.4.0 release notes

Committed 4398109 and pushed to 8.4.x. Thanks!

I've checked all the credit assigned to reviewers and it looks correct.

  • alexpott committed 4398109 on 8.4.x
    Issue #2293697 by Wim Leers, dawehner, Jo Fitzgerald, clemens.tolboom,...
Wim Leers’s picture

This finally unblocked #2800097: Confusing UI: POST URL differs from GET/PATCH/DELETE URL, but UI doesn't indicate this!, which I indicated in #164 was being blocked by this.

It also unblocks a next step: #2851984: Add "create" link template to User entity type annotation, to allow POSTing to /user instead of /entity/user.


So glad this is finally in!

It’s been in reroll hell since March 21 (almost 2 months ago). Before that, it was RTBC on Feb 14. And before that it was RTBC in June 2016! It took a year to get this fixed!

andypost’s picture

Change record said about 8.3 looks it needs update

Wim Leers’s picture

#243: I noticed that myself too, already fixed :)

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.