Problem/Motivation

Discovered by pwolanin & klausi.

STR:

  1. Install D8.
  2. Log out.
  3. Load /node/5. This is a 404.
  4. Log in.
  5. Create node 5 (or anything that is served by /node/5, really).
  6. Log out.
  7. Load /node/5. It's still a 404.

Proposed resolution

When installing modules/themes, we already clear all caches, including the render cache, which also invalidates the Page Cache/reverse proxy. So no problem for newly added routes.

Either:

  • Don't cache non-2xx responses. But that means that sites can be attacked (DDoS) more easily.
  • We should also make sure that 404/403 (generally, non-2xx) responses are invalidated whenever any entity that has a route is saved. Create a "non-2xx-response" cache tag, set it on every non-2xx response, invalidate this cache tag whenever anything happens that may turn a non-2xx in a 2xx. It's okay to invalidate cached non-2xx responses frequently, because they usually have low cache hit ratios anyway.

Remaining tasks

TBD. Unclear if the currently proposed solution is sufficient.

User interface changes

None.

API changes

TBD

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Wim Leers’s picture

Wim Leers’s picture

amateescu’s picture

IMO this is a problem for every entity type, not just nodes :)

Berdir’s picture

Another idea, we could do path based invalidation in the page cache for entities, because after we created them, we know their URL. Not going to be very easy, and it won't work for everything, not sure how to notify reverse proxies and so on...

Also, maybe we want to cache 404/403 for a shorter time in general, in a separate issue?

yched’s picture

shorter 404 / 403 ttl ++

pwolanin’s picture

I think a shorter or different TTL for non-20x is a good idea, but separate issue indeed. The TTL is used by external caches, while here we also need to address Drupal page cache.

pwolanin’s picture

Possibly handle it in ContentEntityBase (check that the entity has a route in the annotation - e.g. not shortcut) and also in router rebuild.

pwolanin’s picture

I'm going to try rolling a patch now.

Wim Leers’s picture

#3: yes, obviously, I was just using a concrete example :)

If we collectively agree that a shorter TTL for non-2xx responses is a good idea, can somebody file an issue for that? :) Thanks!

pwolanin’s picture

pwolanin’s picture

Status: Active » Needs review
FileSize
2.3 KB

This patch is just for the bot - not complete yet.

pwolanin’s picture

Issue tags: +Needs tests
FileSize
3.11 KB
828 bytes

This should be more complete, but needs tests.

Wim Leers’s picture

Looks good already!

Status: Needs review » Needs work

The last submitted patch, 12: 2472281-12.patch, failed testing.

pwolanin’s picture

Status: Needs work » Needs review
FileSize
2.99 KB
6.65 KB
4.23 KB

Fixed some bugs - added test, and test-only patch

dawehner’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: -Needs tests

Seems alright for me!

Wim Leers’s picture

Status: Reviewed & tested by the community » Needs work

Only nits.

  1. +++ b/core/modules/system/src/Tests/Bootstrap/PageCacheTest.php
    @@ -308,6 +309,57 @@ function testPageCacheAnonymousRolePermissions() {
    +    $config = $this->config('system.performance');
    +    $config->set('cache.page.use_internal', 1);
    +    $config->set('cache.page.max_age', 300);
    +    $config->save();
    

    Unnecessary: page caching is enabled by default.

  2. +++ b/core/modules/system/src/Tests/Bootstrap/PageCacheTest.php
    @@ -308,6 +309,57 @@ function testPageCacheAnonymousRolePermissions() {
    +      $this->assertEqual($this->drupalGetHeader('X-Drupal-Cache'), 'MISS');
    +      $cache_tags = explode(' ', $this->drupalGetHeader('X-Drupal-Cache-Tags'));
    +      $this->assertTrue(in_array('4xx-response', $cache_tags));
    

    $this->assertCacheTag('4xx-response');

  3. +++ b/core/modules/system/src/Tests/Bootstrap/PageCacheTest.php
    @@ -308,6 +309,57 @@ function testPageCacheAnonymousRolePermissions() {
    +      // Saving an entity clears 4xx cache tag.
    

    s/clears/invalidates the/

  4. +++ b/core/modules/system/src/Tests/Bootstrap/PageCacheTest.php
    @@ -308,6 +309,57 @@ function testPageCacheAnonymousRolePermissions() {
    +      // Rebuilding the router should clear the 4xx cache tag.
    

    s/clear/invalidate/

The last submitted patch, 15: 2472281-test-only-15.patch, failed testing.

pwolanin’s picture

Status: Needs work » Needs review
FileSize
2.74 KB
6.39 KB
1.76 KB

fixed test.

Wim Leers’s picture

Status: Needs review » Reviewed & tested by the community

Perfect.

The last submitted patch, 19: 2472281-test-only-19.patch, failed testing.

alexpott’s picture

Status: Reviewed & tested by the community » Needs review

Let's wait on a green test.

Wim Leers’s picture

Status: Needs review » Reviewed & tested by the community

Green :)

alexpott’s picture

Status: Reviewed & tested by the community » Fixed

This issue addresses a major bug and is allowed per https://www.drupal.org/core/beta-changes. Committed afdbe0d and pushed to 8.0.x. Thanks!

catch’s picture

Alex beat me to it but +1.

  • alexpott committed afdbe0d on 8.0.x
    Issue #2472281 by pwolanin: 404/403 responses for non-existing nodes are...
znerol’s picture

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.