Problem/Motivation

None of Forum module's responses set cache tags, i.e. none of ForumController.

Uncovered by #2429617: Make D8 2x as fast: Dynamic Page Cache: context-dependent page caching (for *all* users!).

Proposed resolution

Apply the patch in #2429617-57: Make D8 2x as fast: Dynamic Page Cache: context-dependent page caching (for *all* users!), notice how the tests in ForumIndexTest fail. Update ForumController, and perhaps also ForumManager to add cache tags. Note that also list cache tags are missing. Look at \Drupal\book\Controller\BookController::bookRender() for an example.

Remaining tasks

Work on the proposed resolution.

User interface changes

None.

API changes

None.

Comments

wim leers’s picture

Title: Forum responses doesn't set cache tags » Forum responses don't set cache tags
larowlan’s picture

Assigned: Unassigned » larowlan

Sure

borisson_’s picture

Status: Active » Needs review
StatusFileSize
new627 bytes

I forgot to assign this to myself before I started working on this, I attached a patch that should add the correct cache tags. Test passes locally anyway. I have yet to write a specific test to see if the tags added.

borisson_’s picture

StatusFileSize
new1.23 KB
new633 bytes

Adds a test to check this as well.

wim leers’s picture

+++ b/core/modules/forum/src/Controller/ForumController.php
@@ -208,6 +208,9 @@ protected function build($forums, TermInterface $term, $topics = array(), $paren
+        'tags' => \Drupal::entityManager()->getDefinition('node')->getListCacheTags(),

Could you inject this?

ForumController already has this:

  public static function create(ContainerInterface $container) {
    /** @var \Drupal\Core\Entity\EntityManagerInterface $entity_manager */
    $entity_manager = $container->get('entity.manager');
    return new static(
      $container->get('forum_manager'),
      $entity_manager->getStorage('taxonomy_vocabulary'),
      $entity_manager->getStorage('taxonomy_term'),
      $container->get('current_user'),
      $entity_manager->getAccessControlHandler('node'),
      $entity_manager->getFieldMap(),
      $entity_manager->getStorage('node_type'),
      $container->get('renderer')
    );
  }

So most of the facilities are already there :)

borisson_’s picture

StatusFileSize
new2.36 KB
new2.36 KB

Status: Needs review » Needs work

The last submitted patch, 6: forum_responses_don_t-2484619-6.patch, failed testing.

borisson_’s picture

Status: Needs work » Needs review
StatusFileSize
new21.44 KB

Previous patch file was an interdiff. This one should be correct.

Status: Needs review » Needs work

The last submitted patch, 8: forum_responses_don_t-2484619-8.patch, failed testing.

borisson_’s picture

Status: Needs work » Needs review
StatusFileSize
new2.91 KB

This should finally be the correct patch.

larowlan’s picture

We're also using taxonomy hierarchy in the index page - should we be adding tags for that too?

dawehner’s picture

  1. +++ b/core/modules/forum/src/Controller/ForumController.php
    @@ -93,8 +93,10 @@ class ForumController extends ControllerBase {
    -  public function __construct(ForumManagerInterface $forum_manager, VocabularyStorageInterface $vocabulary_storage, TermStorageInterface $term_storage, AccountInterface $current_user, EntityAccessControlHandlerInterface $node_access, array $field_map, EntityStorageInterface $node_type_storage, RendererInterface $renderer) {
    +  public function __construct(ForumManagerInterface $forum_manager, VocabularyStorageInterface $vocabulary_storage, TermStorageInterface $term_storage, AccountInterface $current_user, EntityAccessControlHandlerInterface $node_access, array $field_map, EntityStorageInterface $node_type_storage, RendererInterface $renderer, array $node_list_cache_tags) {
    
    @@ -103,6 +105,7 @@ public function __construct(ForumManagerInterface $forum_manager, VocabularyStor
    +    $this->nodeListCacheTags = $node_list_cache_tags;
    

    This seems to be rather we should fetch on runtime, doesn't it?

  2. +++ b/core/modules/forum/src/Controller/ForumController.php
    @@ -208,6 +212,9 @@ protected function build($forums, TermInterface $term, $topics = array(), $paren
           'action' => $this->buildActionLinks($config->get('vocabulary'), $term),
    

    For that we probably also need the node type list tags or at least the node types cache tags.

wim leers’s picture

borisson_’s picture

StatusFileSize
new4.51 KB
new4.62 KB

I've added the node type list cache tags as dahwehner suggested in #12.2 and as per #12.1 we're now fetching the cache tags on run time.

I'm not sure which cache tag to add for taxonomy hierarchy, is this in the same way we're getting them for nodes / node list?

It feels really wrong to have such a big amount of arguments in the constructor.

dawehner’s picture

Status: Needs review » Needs work
  1. +++ b/core/modules/forum/src/Controller/ForumController.php
    @@ -75,6 +76,16 @@ class ForumController extends ControllerBase {
    +  private $nodeListDefinition;
    ...
    +  private $nodeDefinition;
    

    Let's not use private stupidness.

  2. +++ b/core/modules/forum/src/Controller/ForumController.php
    @@ -111,6 +128,7 @@ public function __construct(ForumManagerInterface $forum_manager, VocabularyStor
    +    $tags = $entity_manager->getDefinition('taxonomy_term')->getListCacheTags();
    

    $tags is not used here.

borisson_’s picture

Status: Needs work » Needs review
StatusFileSize
new1.13 KB
new22.97 KB

Fixed both remakrs from #16.

dawehner’s picture

Status: Needs review » Needs work
+++ b/core/core.services.yml
@@ -875,6 +875,27 @@ services:
+  cache.smart_cache_contexts:
...
+  cache.smart_cache_html:
...
+  smart_cache_subscriber:

We don't want that be part of the patch, don't we?

The last submitted patch, 17: forum_responses_don_t-2484619-17.patch, failed testing.

borisson_’s picture

Status: Needs work » Needs review
StatusFileSize
new4.43 KB

You are absolutely right, looks I should stop writing patches for today :-D. Attached what should be the correct patch.

fabianx’s picture

Assigned: larowlan » wim leers

Assigning to Wim for review.

larowlan’s picture

  1. +++ b/core/modules/forum/src/Controller/ForumController.php
    @@ -93,8 +104,12 @@ class ForumController extends ControllerBase {
    +   *   Node definition object
    ...
    +   *   Node list definition object
    

    I have no idea what either of these are. I think they're the 'The Node entity-type definition' and the 'The Node-type entity-type definition' - can we make the variable names and comments clearer?

  2. +++ b/core/modules/forum/src/Controller/ForumController.php
    @@ -208,6 +228,9 @@ protected function build($forums, TermInterface $term, $topics = array(), $paren
    +        'tags' => Cache::mergeTags($this->nodeDefinition->getListCacheTags(), $this->nodeListDefinition->getListCacheTags()),
    

    I still think this lacks a reference to the term and comments. The number of comments will change the cached build.

wim leers’s picture

Status: Needs review » Needs work

Sorry for taking so long to review.

  1. +++ b/core/modules/forum/src/Controller/ForumController.php
    @@ -75,6 +76,16 @@ class ForumController extends ControllerBase {
       /**
    +   * @var \Drupal\Core\Entity\EntityTypeInterface
    +   */
    +  protected $nodeListDefinition;
    +
    +  /**
    +   * @var \Drupal\Core\Entity\EntityTypeInterface
    +   */
    +  protected $nodeDefinition;
    
    @@ -93,8 +104,12 @@ class ForumController extends ControllerBase {
    +   * @param \Drupal\Core\Entity\EntityTypeInterface $node_definition
    +   *   Node definition object
    +   * @param \Drupal\Core\Entity\EntityTypeInterface $node_list_definition
    +   *   Node list definition object
    

    Comments here need some love.

  2. +++ b/core/modules/forum/src/Controller/ForumController.php
    @@ -103,6 +118,8 @@ public function __construct(ForumManagerInterface $forum_manager, VocabularyStor
    +    $this->nodeDefinition = $node_definition;
    +    $this->nodeListDefinition = $node_list_definition;
    
    @@ -119,7 +137,9 @@ public static function create(ContainerInterface $container) {
    +      $entity_manager->getDefinition('node'),
    +      $entity_manager->getDefinition('node_type')
    
    @@ -208,6 +228,9 @@ protected function build($forums, TermInterface $term, $topics = array(), $paren
         return [
           'action' => $this->buildActionLinks($config->get('vocabulary'), $term),
           'forum' => $build,
    +      '#cache' => [
    +        'tags' => Cache::mergeTags($this->nodeDefinition->getListCacheTags(), $this->nodeListDefinition->getListCacheTags()),
    +      ],
    

    The first one makes sense, the second does not.

    The second was added because of @dawehner's comment earlier:

    For that we probably also need the node type list tags or at least the node types cache tags.

    @dawehner was right that it depends on the node type, but this is currently depending on the *list* of node types. That's wrong. We only need to depend on the node types that are actually depended upon.

    So, in ::buildActionLinks(), we want something like this:

    +        $node_type = $this->nodeTypeStorage->load($type);
             $links[$type] = [
               '#attributes' => ['class' => ['action-links']],
               '#theme' => 'menu_local_action',
               '#link' => [
                 'title' => $this->t('Add new @node_type', [
    -              '@node_type' => $this->nodeTypeStorage->load($type)->label(),
    +              '@node_type' => $node_type->label(),
                 ]),
                 'url' => Url::fromRoute('node.add', ['node_type' => $type]),
               ],
    +          '#cache' => [
    +            'tags' => $node_type->getCacheTags(),
    +          ]
    
  3. +++ b/core/modules/forum/src/Tests/ForumIndexTest.php
    @@ -58,6 +58,8 @@ function testForumIndexStatus() {
    +    $this->assertCacheTag('config:node_type_list');
    

    Then this would be replaced with something like config:node_type.forum

wim leers’s picture

Assigned: wim leers » Unassigned
borisson_’s picture

Status: Needs work » Needs review
StatusFileSize
new5.07 KB
new4.9 KB

Fixes all the remarks made in #23 and #22.1

wim leers’s picture

Status: Needs review » Needs work

#25 fully addresses #23 and #22.1.

That only leaves #23.2:

I still think this lacks a reference to the term and comments. The number of comments will change the cached build.

By this, @larowlan was referring to

    $build = array(
      '#theme' => 'forums',
      '#forums' => $forums,
      '#topics' => $topics,
      '#parents' => $parents,
      '#header' => $header,
      '#term' => $term,
      '#sortby' => $config->get('topics.order'),
      '#forums_per_page' => $config->get('topics.page_limit'),
    );

In there, $forums, $topics, $parents and $term are all arrays of content entities. You basically want to iterate over all of them, and invoke $this->renderer->addCacheableDependency($build, $the_entity_being_iterated) on all of them.

And then, finally, regarding the "number of comments" part of @larowlan's comment, that needs the comment_list cache tag. Which means you want to inject the Comment entity type definition just like the Node entity type definition, and use its list cache tags.

borisson_’s picture

Status: Needs work » Needs review
StatusFileSize
new4.17 KB
new6.11 KB

I added the extra cacheable dependencies. I noticed that $term is not an array but a TermInterface, so I didn't add this in a loop.

I added the comment list cache tag and changed the test so that it looks at at the comment_list cache tag. This might need extra work on the test to check for cache tags that are added trough the new addCacheableDependency calls.

Status: Needs review » Needs work

The last submitted patch, 27: forum_responses_don_t-2484619-27.patch, failed testing.

wim leers’s picture

+++ b/core/modules/forum/src/Controller/ForumController.php
@@ -205,9 +229,23 @@ protected function build($forums, TermInterface $term, $topics = array(), $paren
+    foreach ($forums as $forum) {
+      $this->renderer->addCacheableDependency($build, $forum);
+    }

The exceptions in the test runner show that this is also not an array, but a single entity.

borisson_’s picture

Status: Needs work » Needs review
StatusFileSize
new829 bytes
new6.15 KB

Not happy with this solution but this does makes ForumTest pass locally.

wim leers’s picture

Can $topics be NULL? That's the only reason I can think of for that strange failure. Can't you then just change the logic that provides $topics so that it defaults to the empty array instead?

borisson_’s picture

$topics already defaults to an empty array.
protected function build($forums, TermInterface $term, $topics = array(), $parents = array(), $header = array()) {

wim leers’s picture

Then... huh? I can't use a debugger ATM, not at my workstation.

wim leers’s picture

Status: Needs review » Reviewed & tested by the community
StatusFileSize
new6.44 KB
new1.26 KB

I did the necessary debugging to figure out the answer to #30/#31/#32/#33.

The root cause was silly & simple: in one branch, \Drupal\forum\Controller\ForumController::forumPage() was setting $topics to the empty string. That caused it.

That sole tiny fix doesn't prevent me from RTBC'ing this patch IMHO, so: RTBC! :)

alexpott’s picture

Status: Reviewed & tested by the community » Needs review
+++ b/core/modules/forum/src/Controller/ForumController.php
@@ -205,9 +229,23 @@ protected function build($forums, TermInterface $term, $topics = array(), $paren
+    foreach ($forums as $forum) {
+      $this->renderer->addCacheableDependency($build, $forum);
+    }
+    foreach ($topics as $topic) {
+      $this->renderer->addCacheableDependency($build, $topic);
+    }
+    foreach ($parents as $parent) {
+      $this->renderer->addCacheableDependency($build, $parent);
+    }
+    $this->renderer->addCacheableDependency($build, $term);

Is there test coverage of this?

wim leers’s picture

Status: Needs review » Needs work
Issue tags: +Needs tests

Oops, you're right, those bits are missing from test coverage. Sorry.

lauriii’s picture

Status: Needs work » Needs review
Issue tags: -Needs tests
StatusFileSize
new7.48 KB
new1.66 KB
wim leers’s picture

Status: Needs review » Reviewed & tested by the community

Awesome, thanks!

alexpott’s picture

Category: Task » Bug report
Status: Reviewed & tested by the community » Fixed

Not exposing the correct cache tags is a bug no? This issue addresses a major bug and is allowed per https://www.drupal.org/core/beta-changes. Committed 41560b3 and pushed to 8.0.x. Thanks!

  • alexpott committed 41560b3 on 8.0.x
    Issue #2484619 by borisson_, Wim Leers, lauriii, larowlan, dawehner:...
wim leers’s picture

Hehe, right :)

Thanks!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.