Problem/Motivation

The 'url' cache context for views block displays is sometimes missing when it shouldn't be because \Drupal\views\Plugin\views\display\DisplayPluginBase::buildRenderable() only bubbles the context cachablity metadata when it has been asked to create a render cache for the output.

This can lead to problems where empty view blocks get cached.

Steps to reproduce

This started out as a long adventure into why empty view blocks get cached, so here are some steps to reproduce:

Setup

(note that I exported and attached my test site's configuration in #2 — this could save you time)

  1. Install Drupal 8.0.x using the minimal install profile.
  2. Enable field_ui, views, views_ui modules.
  3. Go to /admin/structure/types/add, add a newcontent type:
    • Name = Test
    • Publishing options:
      • Promoted to front page = FALSE
    • (leave the rest of the fields at their default value)
  4. Go to admin/structure/types/manage/test/display and disable the body field (i.e.: set it's Format = - Hidden -).
  5. Go to /node/add/test, add a first new piece of content:
    • Title = First test
    • Body = Lorem ipsum
    • (leave the rest of the fields at their default value)
  6. Go to /node/add/test, add a second new piece of content:
    • Title = Second test
    • Body = Dolor sit
    • (leave the rest of the fields at their default value)
  7. Go to /admin/structure/views/add, add a new view:
    • View basic information:
      • View name = "Tests"
      • (leave the rest of the fields at their defaultvalue)
    • View settings:
      • Show Content of type Test sorted by Unsorted
    • Page settings
      • (leave the rest of the fields at their default value)
    • Block settings
      • Create a block = TRUE
      • Block title = Tests
      • Block display settings:
        • Display format: Unformatted list of Fields
    • (leave the rest of the fields at their default value)
    • Click Save and edit
    • Under Fields, remove the default Content: Title field, and add a Content: Body field (the default settings are fine).
    • Under Advanced, add a Contextual Filter. Select Node ID, and click
      Apply (all displays):
      • When the filter value is NOT available:
        • Default actions: Provide default value:
          • Type = Content ID from URL
      • (leave the rest of the fields at their default value)
    • Save the view.
  8. Go to admin/structure/block
    • In Footer, click Place block. Find the Tests block from the category Lists (Views), and click Place block. Save using the default settings.
    • Click Save blocks
Test 1
  1. Clear all caches: drush -y cr
  2. Go to /node.
  3. Go to /node/1.
    • Expcected behaviour: A block is displayed in the footer with the contents of the body field ("Lorem ipsum").
    • Actual behaviour: No block is displayed.
  4. Go to /node/2.
    • Expcected behaviour: A block is displayed in the footer with the contents of the body field ("Dolor sit").
    • Actual behaviour: No block is displayed.
Test 2
  1. Clear all caches: drush -y cr
  2. Go to /node/1. Note the block is displayed in the footer with the contents of the body field ("Lorem ipsum").
  3. Go to /node/2. Note the block is displayed in the footer with the contents of the body field ("Dolor sit").
Test 3
  1. In settings[.local].php, disable the render cache by adding the following lines:
    $settings['container_yamls'][] = DRUPAL_ROOT . '/sites/development.services.yml';
    $settings['cache']['bins']['render'] = 'cache.backend.null';
    
  2. Clear all caches: drush -y cr
  3. Go to /node.
  4. Go to /node/1. Note the block is displayed in the footer with the contents of the body field ("Lorem ipsum").
  5. Go to /node/2. Note the block is displayed in the footer with the contents of the body field ("Dolor sit").
Test 4
  1. Remove the lines from settings[.local].php you added in step 3, if applicable. Re-run Test 1 to confirm the bug occurs.
  2. Apply the following patch:
    diff --git a/core/modules/views/src/Plugin/Block/ViewsBlock.php b/core/modules/views/src/Plugin/Block/ViewsBlock.php
    index 20dbace..530d9e3 100644
    --- a/core/modules/views/src/Plugin/Block/ViewsBlock.php
    +++ b/core/modules/views/src/Plugin/Block/ViewsBlock.php
    @@ -127,4 +127,15 @@ public function getMachineNameSuggestion() {
         return 'views_block__' . $this->view->storage->id() . '_' . $this->view->current_display;
       }
    
    +  /**
    +   * {@inheritdoc}
    +   */
    +  public function getCacheContexts() {
    +    $contexts = parent::getCacheContexts();
    +
    +    $contexts[] = 'url';
    +
    +    return $contexts;
    +  }
    +
     }
    
  3. Clear all caches: drush -y cr
  4. Go to /node.
  5. Go to /node/1. Note the block is displayed in the footer with the contents of the body field ("Lorem ipsum").
  6. Go to /node/2. Note the block is displayed in the footer with the contents of the body field ("Dolor sit").
  7. Proposed resolution

    Always pass the cachablity metadata; but if we're not supposed to cache the view's output, remove the cache tags, leaving the rest of the metadata to bubble.

    Remaining tasks

    1. Figure out what's causing empty blocks to be cached.
    2. Write a patch
    3. Review and feedback
    4. RTBC.
    5. Commit.

    User interface changes

    None.

    API changes

    None.

    Data model changes

    None.

Comments

mparker17 created an issue. See original summary.

mparker17’s picture

StatusFileSize
new56.72 KB

Here is an export of my test site's config, for fast debugging.

mparker17’s picture

Issue tags: +Render Cache, +Regression

Tagging as related to the render cache. I'm pretty sure this worked in D7 / Views 7.x-3.x so I'm tagging as a regression also.

mparker17’s picture

Issue summary: View changes

I should note the config export in the setup section of the steps to reproduce

mparker17’s picture

Issue summary: View changes
mparker17’s picture

Title: The 'url' cache context for views block displays is sometimes missing when it shouldn't be » Views cache contexts are lost (and thus do not bubble) when rendering a view's block display.
Assigned: mparker17 » Unassigned
Issue summary: View changes
Status: Active » Needs review
Related issues: +#2633388: Document why ViewsBlock::build() explicitly asks for an un-cached view., +#2381277: Make Views use render caching and remove Views' own "output caching"
StatusFileSize
new1.81 KB

Ok, so after a bit of debugging and chatting with @dawehner, it looks like \Drupal\views\Plugin\views\display\DisplayPluginBase::buildRenderable() only includes the context cachablity metadata when it has been asked to create a render cache for the output, which isn't correct.

Instead, we always want to pass the cachablity metadata; but if we're not supposed to cache the view's output, remove the cache tags, leaving the rest of the metadata to bubble.

dawehner’s picture

No we should really have some form of test coverage, just to ensure this will never break again.

fabianx’s picture

Priority: Normal » Major

This is major, great catch!

fabianx’s picture

Status: Needs review » Needs work
Issue tags: +Needs tests, +D8 cacheability
+++ b/core/modules/views/src/Plugin/views/display/DisplayPluginBase.php
@@ -2335,13 +2335,17 @@ public function buildRenderable(array $args = [], $cache = TRUE) {
+    // Thus, we add the cachability metadata first, then modify / remove the
+    // cache keys depending on the $cache argument.
+    $this->applyDisplayCachablityMetadata($this->view->element);

I would probably do it after the if check.

mparker17’s picture

@Fabianx: I initially did it after the if check, but DisplayPluginBase::applyDisplayCachablityMetadata() copies over cache tags as well, meaning it puts back the cache tags after the else part removes them.

fabianx’s picture

#10: Cache tags should not be removed - that we need to discuss separately however.

If the view is cached in dynamic page cache, why would we not want to expire it via its tags?

mparker17’s picture

Status: Needs work » Needs review
StatusFileSize
new1.34 KB
new3.21 KB

Here is a patch which adds a test to compare the cache contexts with and without render caching. Dunno if this is the best way to test this, but trying to manually add a cache context ($view->addCacheContext('views_test_cache_context')) didn't work — manually-added cache contexts showed up in the output with and without caching enabled.

Feedback welcome.

mparker17’s picture

StatusFileSize
new1.1 KB
new2.77 KB

... and apparently I attached a half-finished patch... let's try that again.

sagesolutions’s picture

Hi mparker17,

I had the exact same issue which was driving me bananas. I used updated my site with views_cache_contexts-2633308-13.patch and it fixed my issue.

Thanks so much!
Mike

dawehner’s picture

The patch indeed fixes the issue already. One thing I'm wondering is, should we have some kind of regression test coverage for the empty block problem, so the disappearing block?
Yes, the cache contexts are there, but it seems to be part of a detail for me, so having a test coverage which basically does exactly what is described in the issue summary would be neat and super helpful IMHO

alayham’s picture

Status: Needs review » Reviewed & tested by the community

Tested the patch and resolved a complex problem I reported in #2647818: Views blocks with taxonomy arguments disappear for no reason

wim leers’s picture

Issue tags: -Needs tests

Looks sensible.

kclarkson’s picture

I am confirming that the attached patch has fixed my issue from #2650338: Contextual Views Blocks Disappearing

+1 RTBC

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 13: views_cache_contexts-2633308-13.patch, failed testing.

dawehner’s picture

Status: Needs work » Reviewed & tested by the community

Back to RTBC

gunwald’s picture

Seems to solve my issue from https://www.drupal.org/node/2655522 too, I am happy again.

alexpott’s picture

Status: Reviewed & tested by the community » Needs work

IN #15 @dawehner suggested some integration tests based on the excellent steps to reproduce in the issue summary. Given the number of related issues I think this is worth it.

dawehner’s picture

I'm looking into writing the test coverage.

dawehner’s picture

Status: Needs work » Needs review
StatusFileSize
new10.86 KB
new4.16 KB
dawehner’s picture

StatusFileSize
new7.98 KB

Here is a patch which will fail

Status: Needs review » Needs work

The last submitted patch, 25: 2633308-24-fail.patch, failed testing.

dawehner’s picture

Status: Needs work » Needs review

Back to needs review, failures are as expected

wim leers’s picture

Status: Needs review » Needs work
  1. +++ b/core/modules/user/config/schema/user.views.schema.yml
    @@ -56,10 +56,6 @@ views.argument_default.current_user:
    -views.argument_default.node:
    -  type: boolean
    -  label: 'Content ID from URL'
    

    This seems unrelated? Perhaps even unwanted?

  2. +++ b/core/modules/views/src/Plugin/views/display/DisplayPluginBase.php
    @@ -2335,13 +2335,17 @@ public function buildRenderable(array $args = [], $cache = TRUE) {
    +    // of cachablity metadata (e.g.: cache contexts), so they can bubble up.
    +    // Thus, we add the cachability metadata first, then modify / remove the
    

    s/cachability/cacheability/

  3. +++ b/core/modules/views/src/Tests/Plugin/CacheWebTest.php
    @@ -84,4 +84,15 @@ public function testCacheOutputOnPage() {
    +    $view = Views::getView('test_display');
    +
    +    $uncached_block = $view->buildRenderable('block_1', [], FALSE);
    +    $cached_block = $view->buildRenderable('block_1', [], TRUE);
    +    $this->assertEqual($uncached_block['#cache']['contexts'], $cached_block['#cache']['contexts'], 'Cache contexts are the same when you render the view cached and uncached.');
    

    This test makes sense. Simple & clear.

  4. +++ b/core/modules/views/src/Tests/RenderCacheWebTest.php
    @@ -0,0 +1,79 @@
    +class RenderCacheWebTest extends ViewTestBase {
    

    I'm not entirely sure what this test is testing?

dawehner’s picture

Status: Needs work » Needs review
StatusFileSize
new10.86 KB
new1 KB

This seems unrelated? Perhaps even unwanted?

This view is the first one in core which uses views.argument_default.node in config.

Now something fun exists in core. This is defined twice: once in node.views.schema.yml and once in user.views.schema.yml, which overrides the first one, with a wrong schema definition,
which makes it impossible to write a passing test.

I'm not entirely sure what this test is testing?

Well, the bug of this issue. I hope its fine to do that :)

There we go.

tannerjfco’s picture

Status: Needs review » Reviewed & tested by the community

We were experiencing this issue and applied the patch in #29. We have been running this in production and the issue seems to be resolved. Since it seems to fix the original issue and we have passing tests, I'm gonna go ahead and RTBC. If there's remaining work feel free to update accordingly.

alayham’s picture

Patch #29 works well.
Looking forward to remove it from my patch file.

wim leers’s picture

#29 Thanks for the explanation!

Well, the bug of this issue.

My only concern there is that both the test name and the description are very generic. I think it'd be helpful for our future selves if it'd describe more explicitly what it's testing. But it's only a nit, so leaving at RTBC.

alexpott’s picture

Status: Reviewed & tested by the community » Needs work
  1. +++ b/core/modules/user/config/schema/user.views.schema.yml
    @@ -56,10 +56,6 @@ views.argument_default.current_user:
    -views.argument_default.node:
    -  type: boolean
    -  label: 'Content ID from URL'
    -
    

    Let's get a follow up to add a specific test for this then. Nice to have it fixed though.

  2. +++ b/core/modules/views/src/Tests/RenderCacheWebTest.php
    @@ -0,0 +1,79 @@
    + * Tests render caching on an actual site.
    

    I don't mind about the class name but the test description could be a bit better. I left pondering what an actual site means in this context. How about we change this to
    Tests render caching of blocks provided by views.

dawehner’s picture

Status: Needs work » Needs review
StatusFileSize
new10.87 KB
new484 bytes
wim leers’s picture

Status: Needs review » Reviewed & tested by the community
alexpott’s picture

Status: Reviewed & tested by the community » Fixed

Committed cf2c7a4 and pushed to 8.0.x and 8.1.x. Thanks!

  • alexpott committed 130f8c1 on 8.1.x
    Issue #2633308 by dawehner, mparker17, Wim Leers, Fabianx: Views cache...

  • alexpott committed cf2c7a4 on
    Issue #2633308 by dawehner, mparker17, Wim Leers, Fabianx: Views cache...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.