Problem/Motivation

A consequence of this bug in HEAD is that any route whose access check result varies per role, does not get the cache contexts/tags/max-age set on the response, and hence the page cache doesn't include the necessary cache tags. So when permissions are granted or revoked to the anonymous user, anonymous users will continue to get the stale cached pages from the page cache. (Also see #15.)

Blocks #606840: Enable internal page cache by default.

See #606840-26: Enable internal page cache by default in particular, but also the other comments up to and including comment 31 on that issue.

Proposed resolution

Ensure route access checking's access result's cacheability metadata is applied to the response, hence ensuring proper invalidation of the page cache.

Remaining tasks

None.

User interface changes

None.

API changes

None.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Wim Leers’s picture

Status: Active » Needs review
Issue tags: +Needs tests
FileSize
14.94 KB
Wim Leers’s picture

Wim Leers’s picture

Issue summary: View changes

Status: Needs review » Needs work

The last submitted patch, 1: route_access_result_cacheability-2458349-1.patch, failed testing.

Wim Leers’s picture

Title: Route's access result's cacheability not applied to the response's cacheability » [PP-1] Route's access result's cacheability not applied to the response's cacheability
Status: Needs work » Needs review
FileSize
14.42 KB
2.11 KB

Minor fixes.

But the test failures themselves can only be fixed by making render arrays not set incorrect max-age values. Currently, blocks default to max-age zero, even when they're actually cacheable. The presence of any such block makes the page uncacheable (as expected). AFAICT all test failures are due to tests verifying page cache entries unable to find those page cache entries due to incorrect usage of max-age zero.

Blocks setting the wrong max-age will be fixed in #2428805: Remove the ability to configure a block's cache contexts

Status: Needs review » Needs work

The last submitted patch, 5: route_access_result_cacheability-2458349-5.patch, failed testing.

Wim Leers’s picture

Title: [PP-1] Route's access result's cacheability not applied to the response's cacheability » Route's access result's cacheability not applied to the response's cacheability
Wim Leers’s picture

Status: Needs work » Needs review
FileSize
13.48 KB
3.06 KB

Too many things are currently still setting max-age zero, making the addition of the "max age zero -> don't cache" page cache response policy infeasible.
Besides being infeasible, it's also out of scope for this issue. So, removing that from this patch.
(It may be the long-term ideal, but that doesn't make this issue the right place to do it.)

Also 2 small changes because #2428703: Add a 'user.permissions' cache context (was: "Should cache contexts be able to associate a cache tag?") has landed.

This should makes the patch (almost) green. Next: tests.

Status: Needs review » Needs work

The last submitted patch, 8: route_access_result_cacheability-2458349-8.patch, failed testing.

Wim Leers’s picture

Status: Needs work » Needs review
Issue tags: -Needs tests
FileSize
18.14 KB
4.73 KB

Added test coverage.

Next: fixing #8's fails.

Wim Leers’s picture

FileSize
16.7 KB
2.22 KB

Removed all the max-age stuff, that was interfering with the page cache. This is in line with #8: the page cache currently just isn't designed to deal with a bubbling max-age. Therefore, we limit ourselves to cache contexts & cache tags.

EDIT: we have #2352009: [pp-3] Bubbling of elements' max-age to the page's headers and the page cache for doing exactly that.

The last submitted patch, 10: route_access_result_cacheability-2458349-9.patch, failed testing.

Wim Leers’s picture

FileSize
17.36 KB
3.3 KB

Clearer & more complete test coverage.

dawehner’s picture

  1. +++ b/core/lib/Drupal/Core/EventSubscriber/DefaultExceptionHtmlSubscriber.php
    @@ -111,8 +112,10 @@ protected function makeSubrequest(GetResponseForExceptionEvent $event, $url, $st
    -        // Persist the 'exception' attribute to the subrequest.
    +        // Persist the 'exception' and access result attributes to the
    +        // subrequest.
             $sub_request->attributes->set('exception', $request->attributes->get('exception'));
    +        $sub_request->attributes->set(AccessAwareRouterInterface::ACCESS_RESULT, $request->attributes->get(AccessAwareRouterInterface::ACCESS_RESULT));
    

    The documentation was and is still not helpful at all. Yes, we see what the code is doing, but there is nothing about the why.

  2. +++ b/core/lib/Drupal/Core/EventSubscriber/FinishResponseSubscriber.php
    @@ -139,6 +159,30 @@ public function onRespond(FilterResponseEvent $event) {
    +      $existing_cache_contexts = explode(' ', $response->headers->get('X-Drupal-Cache-Contexts'));
    +      $cache_contexts = Cache::mergeTags($existing_cache_contexts, $cache_contexts);
    

    I'm glad that Cache::mergeTags works the same as Cache::mergeContexts

  3. +++ b/core/modules/system/tests/modules/router_test_directory/src/TestControllers.php
    @@ -84,6 +84,19 @@ public function test10() {
    +        '#markup' => "test18",
    

    nitpick hell: use single quotes ...

Wim Leers’s picture

Priority: Major » Critical
Issue tags: +Security, +Needs Drupal 8 critical triage

I believe this is critical for similar reasons as why #2428703: Add a 'user.permissions' cache context (was: "Should cache contexts be able to associate a cache tag?") was critical: until we have this, any access result that's invalidated by a certain cache tag, or that is invalidated by a change in the permissions for a role, will not be invalidated. Therefore, when page caching is enabled, the anonymous user will continue to have access to things that they shouldn't have access to even after e.g. the permissions for the anonymous user have changed.

Wim Leers’s picture

Issue summary: View changes
FileSize
17.4 KB
1.82 KB

#14

  1. I just followed the same style as for the pre-existing "exception" case. But, totally agreed. Fixed.
  2. :)
  3. :D Fixed :)
dawehner’s picture

Status: Needs review » Needs work
+++ b/core/lib/Drupal/Core/EventSubscriber/FinishResponseSubscriber.php
@@ -139,6 +159,30 @@ public function onRespond(FilterResponseEvent $event) {
+
+    // X-Drupal-Cache-Contexts
+    $cache_contexts = $cacheable_access_result->getCacheContexts();
+    if ($response->headers->has('X-Drupal-Cache-Contexts')) {
+      $existing_cache_contexts = explode(' ', $response->headers->get('X-Drupal-Cache-Contexts'));
+      $cache_contexts = Cache::mergeTags($existing_cache_contexts, $cache_contexts);
+    }
+    $response->headers->set('X-Drupal-Cache-Contexts', implode(' ', $this->cacheContexts->optimizeTokens($cache_contexts)));

Alright, you didn't got me ... $cache_contexts = Cache::mergeTags ... its certainly using the wrong method. In general I'm curious whether we at any time had more than 2 contexts/tags we merged at the same time, ... the API is a bit sad as its documentation is not optimal.

Wim Leers’s picture

Status: Needs work » Needs review
FileSize
17.41 KB
1.06 KB

UGH, c/p error.

Yes, we merge >2 tags in several places. E.g. \Drupal\block\BlockViewBuilder::viewMultiple(). For ::mergeContexts(), I just made it analogous with ::mergeTags().

dawehner’s picture

Status: Needs review » Reviewed & tested by the community

Yes, we merge >2 tags in several places. E.g. \Drupal\block\BlockViewBuilder::viewMultiple(). For ::mergeContexts(), I just made it analogous with ::mergeTags().

Well, we could have one method doing multiple and the default is just a simlpe wrapper, but these are just ideas.

Wim Leers’s picture

#19: good ideas, but having them as separate methods keeps the door open for smarter validation in the future — they're conceptually different things, they're just both represented as strings.

Fabianx’s picture

RTBC + 1

Fabianx’s picture

+++ b/core/lib/Drupal/Core/EventSubscriber/FinishResponseSubscriber.php
@@ -139,6 +159,30 @@ public function onRespond(FilterResponseEvent $event) {
+  protected function updateDrupalCacheHeaders(Response $response, CacheableInterface $cacheable_access_result) {
+    // X-Drupal-Cache-Tags
...
+    // X-Drupal-Cache-Contexts
+    $cache_contexts = $cacheable_access_result->getCacheContexts();

For the record:

I am not yet happy about the special casing of the headers here.

I think we should always set the headers in the response subscriber and never in the main ContentHtml Thing, but that is out of scope for this particular issue, so should be a follow-up.

---

Still RTBC

dawehner’s picture

#19: good ideas, but having them as separate methods keeps the door open for smarter validation in the future — they're conceptually different things, they're just both represented as strings.

I simply think you don't understand me at all.

Wim Leers’s picture

Talked to @dawehner in IRC. Conclusion at #2454643-5: Optimize cacheability bubbling (Cache::mergeTags(), ::mergeContexts(), BubbleableMetadata), which is a more appropriate issue to further handle that.

Let's keep this focused on route's access result handling.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 18: route_access_result_cacheability-2458349-18.patch, failed testing.

dawehner’s picture

Let's keep this focused on route's access result handling.

Yeah, no question, all I wanted is to throw out some general thoughts. Sorry

The last submitted patch, 18: route_access_result_cacheability-2458349-18.patch, failed testing.

Wim Leers’s picture

Status: Needs work » Reviewed & tested by the community
FileSize
17.41 KB

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 29: route_access_result_cacheability-2458349-29.patch, failed testing.

Wim Leers’s picture

Status: Needs work » Reviewed & tested by the community
FileSize
18.82 KB
3.68 KB

Broken by #2453059: Set default render cache contexts: 'theme' + 'languages:' . LanguageInterface::TYPE_INTERFACE — it introduced new failures. Should be green again now.

Fabianx’s picture

RTBC + 1

alexpott’s picture

Status: Reviewed & tested by the community » Fixed

This issue addresses a critical bug and is allowed per https://www.drupal.org/core/beta-changes. Committed 6c3303d and pushed to 8.0.x. Thanks!

+++ b/core/lib/Drupal/Core/EventSubscriber/FinishResponseSubscriber.php
@@ -139,6 +159,30 @@ public function onRespond(FilterResponseEvent $event) {
+    // X-Drupal-Cache-Tags
+    $cache_tags = $cacheable_access_result->getCacheTags();
+    if ($response->headers->has('X-Drupal-Cache-Tags')) {
+      $existing_cache_tags = explode(' ', $response->headers->get('X-Drupal-Cache-Tags'));
+      $cache_tags = Cache::mergeTags($existing_cache_tags, $cache_tags);
+    }
+    $response->headers->set('X-Drupal-Cache-Tags', implode(' ', $cache_tags));
+
+    // X-Drupal-Cache-Contexts
+    $cache_contexts = $cacheable_access_result->getCacheContexts();
+    if ($response->headers->has('X-Drupal-Cache-Contexts')) {
+      $existing_cache_contexts = explode(' ', $response->headers->get('X-Drupal-Cache-Contexts'));
+      $cache_contexts = Cache::mergeContexts($existing_cache_contexts, $cache_contexts);
+    }
+    $response->headers->set('X-Drupal-Cache-Contexts', implode(' ', $this->cacheContexts->optimizeTokens($cache_contexts)));

I think it is a shame to have to explode and reset these but I discussed with @Wim Leers in IRC and he couldn't see a way around this.

  • alexpott committed 6c3303d on 8.0.x
    Issue #2458349 by Wim Leers: Route's access result's cacheability not...
Wim Leers’s picture

Thanks! This has unblocked #2453341: Contact forms don't have necessary cache contexts & tags; flood control should work on validate, not on view.


Relevant part of the IRC discussion mentioned in #33 for future reference:

15:47:25 alexpott: WimLeers: https://www.drupal.org/node/2458349 feels odd
15:47:26 Druplicon: https://www.drupal.org/node/2458349 => Route's access result's cacheability not applied to the response's cacheability #2458349: Route's access result's cacheability not applied to the response's cacheability => 32 comments, 3 IRC mentions
15:47:46 alexpott: WimLeers: why are we unpacking and then repacking the headers
15:48:25 alexpott: WimLeers: shouldn't the response have access to this information earlier?
15:49:29 WimLeers: Fabianx-screen: ^^
15:50:12 WimLeers: alexpott: a route's controller can return a Response object itself. We don't want to put burden on every single route controller to set the route's access result on the response.
15:51:03 WimLeers: alexpott: But when the route controller returns a render array, then the HtmlRenderer turns the render array (which includes cache tags) into a Response (which includes a X-Drupal-Cache-Tags header)
15:51:29 WimLeers: alexpott: so in that case, we need to detect the presence of the existing data and merge in the route access result's cacheability metadata
mlhess left the room (quit: Remote host closed the connection). (15:51:47)
mlhess [~mlhess@drupal.org/user/102818/view] entered the room. (15:52:09)
15:52:59 WimLeers: alexpott: Simply put: the controller is responsible for the cacheability metadata for the response's contents. The Finish subscriber takes care of the cacheability metadata that happened during access checking while performing routing. So every single response will get route access checking cacheability metadata propagated to the response for free.
15:53:11 WimLeers: alexpott: Hope that made sense.
15:53:22 alexpott: WimLeers: I get that
15:54:37 WimLeers: alexpott: so if a route controller can do `return new Response('hello world')`, I don't see how we could do "shouldn't the response have access to this information earlier?"
15:55:43 alexpott: WimLeers: yeah I see
Fabianx’s picture

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.