Problem/Motivation

As part of #2124957: Replace 'content' cache tag with 'rendered' and use it sparingly, we added extensive test coverage for the most important content entity types' usage of cache tags in #2188565: Test coverage for Comment, Custom Block, Node, Taxonomy Term and User entity cache tags.

Shortcut is one of the last three content entity types (the other two are at #2241229: Aggregator Feed & Item content entity types, and AggregatorFeedBlock should use cache tags) that don't yet use cache tags yet when rendering content.
#2217749: Entity base class should provide standardized cache tags and built-in invalidation added standardized cache tags to all entities, so we can use those.

This also prevents #2254865: toolbar_pre_render() runs on every page and is responsible for ~15ms/17000 function calls from yielding a bigger performance win.

Proposed resolution

  1. Don't assign Shortcut-specific cache tags, use the associated ShortcutSet's cache tags instead. Shortcuts are never rendered on their own, and there are few shortcuts within a shortcut set, so that'd be fine.
  2. Provide full test coverage for the above, by subclassing EntityCacheTagsTestBase.
  3. This is the first entity that subclasses EntityCacheTagsTestBase which does not use the typical <entity type>:<entity ID> cache tags, so this requires EntityCacheTagsTestBase to be updated to use EntityInterface::getCacheTag().
  4. This is the first entity that subclasses EntityCacheTagsTestBase which does not have a view builder, which also requires adjustments.
  5. Let shortcut_renderable_links() assign those shortcuts.

Remaining tasks

None.

User interface changes

None.

API changes

None.

Original report by @Wim Leers

As part of #2124957: Replace 'content' cache tag with 'rendered' and use it sparingly, we added extensive test coverage for the most important content entity types' usage of cache tags in #2188565: Test coverage for Comment, Custom Block, Node, Taxonomy Term and User entity cache tags.

Shortcut is one of the last three content entity types (the other two are at #2241229: Aggregator Feed & Item content entity types, and AggregatorFeedBlock should use cache tags) that don't yet use cache tags yet when rendering content.
#2217749: Entity base class should provide standardized cache tags and built-in invalidation added standardized cache tags to all entities, so we can use those.

Let's use those cache tags for rendered shortcuts and add basic test coverage.

This will help in making the toolbar more cacheable :)

Unfortunately, AFAICT there is no test coverage yet for rendered shortcuts (shortcut_renderable_links()), and definitely not for shortcuts as part of the toolbar. So either that will need to be done in this issue, or this will need to be committed without test coverage (which would still be a step forward).

Comments

wim leers’s picture

Priority: Minor » Normal
Status: Active » Needs review
StatusFileSize
new1.36 KB
wim leers’s picture

Title: Shortcut content entity types should use cache tags » Shortcut/ShortcutSet entity types should use cache tags

Shortcuts are always rendered in a Shortcut Set. Just like Menu Links are always rendered in a Menu.

Therefore, it may make sense to discard Shortcuts' own cache tags entirely and just always make them refer to the Shortcut Set's cache tags.

i.e. something like this in MenuLink::postSave() could also be applied in Shortcut::postSave():

    Cache::invalidateTags(array('menu' => $this->menu_name));
    if (isset($this->original) && $this->menu_name != $this->original->menu_name) {
      Cache::invalidateTags(array('menu' => $this->original->menu_name));
    }
wim leers’s picture

Also see #2241291-1: Regression: menu link-specific cache tags, for the other half of what needs to happen for #2.

wim leers’s picture

Assigned: Unassigned » wim leers
Issue tags: +sprint
Related issues: +#2254865: toolbar_pre_render() runs on every page and is responsible for ~15ms/17000 function calls
StatusFileSize
new17 KB

In #2254865: toolbar_pre_render() runs on every page and is responsible for ~15ms/17000 function calls, we could render cache shortcuts, but only if this lands, otherwise old shortcuts would be served to the end user in the toolbar. So let's get this done.

I had to update \Drupal\system\Tests\Entity\EntityCacheTagsTestBase, because Shortcut is a bit of a special snowflake: it has no view builder, and EntityCacheTagsTestBase was assuming that every entity it'd test would have that. EntityCacheTagsTestBase was also assuming that an entity's cache tag would be based on that entity's entity type, but Shortcut and MenuLink are the special snowflakes in that regard — fortunately we now have EntityInterface::getCacheTag(), so that's easily solved.

Status: Needs review » Needs work

The last submitted patch, 4: shortcut_cache_tags-2241235-4.patch, failed testing.

wim leers’s picture

Status: Needs work » Needs review
StatusFileSize
new14.63 KB
new1.03 KB

Silly mistake.

Status: Needs review » Needs work

The last submitted patch, 6: shortcut_cache_tags-2241235-6.patch, failed testing.

wim leers’s picture

Status: Needs work » Needs review
StatusFileSize
new15.6 KB
new1.71 KB

#6 failed because of a hilarious bug in current D8 HEAD: if you go to admin/config/user-interface/shortcut/add-set, reload that page a few times (say 10 times), then you've now created 10 duplicates of each shortcut in the "default" shortcut set. But those shortcuts are not associated with anything. Because when you go to that page, it'll automatically duplicate those shortcuts, even if you don't actually create a new shortcut set!
Because of this silly behavior, no cache tags could be generated for those duplicated shortcuts. Simply duplicating those shortcuts after the shortcut set is *actually* created fixes the problem :)

wim leers’s picture

Priority: Normal » Major

This prevents better cacheability of shortcuts, and prevents #2254865: toolbar_pre_render() runs on every page and is responsible for ~15ms/17000 function calls from making a bigger improvement. Increasing priority accordingly.

wim leers’s picture

StatusFileSize
new17.5 KB
new1.08 KB

Oops, #6 and #8 didn't include ShortcutCacheTagsTest… restoring that file. Plus, #2099131: Use #pre_render pattern for entity render caching broke this. Rerolled.

wim leers’s picture

Issue summary: View changes

An updated issue summary should help with reviewing.

amateescu’s picture

  1. +++ b/core/modules/shortcut/shortcut.module
    @@ -299,12 +300,15 @@ function shortcut_renderable_links($shortcut_set = NULL) {
    +    $all_cache_tags[] = $shortcut->getCacheTag();
    
    @@ -314,6 +318,9 @@ function shortcut_renderable_links($shortcut_set = NULL) {
    +      '#cache' => array(
    +        'tags' => NestedArray::mergeDeepArray($all_cache_tags),
    +      ),
    

    Isn't a += enough here?

  2. +++ b/core/modules/shortcut/src/Entity/Shortcut.php
    @@ -124,9 +129,26 @@ public static function preCreate(EntityStorageInterface $storage, array &$values
    -    $url = Url::createFromPath($this->path->value);
    -    $this->setRouteName($url->getRouteName());
    -    $this->setRouteParams($url->getRouteParameters());
    +    if ($this->isNew() || !empty($this->path->value)) {
    +      $url = Url::createFromPath($this->path->value);
    +      $this->setRouteName($url->getRouteName());
    +      $this->setRouteParams($url->getRouteParameters());
    +    }
    

    Why is this change needed?

  3. +++ b/core/modules/shortcut/src/Entity/Shortcut.php
    @@ -124,9 +129,26 @@ public static function preCreate(EntityStorageInterface $storage, array &$values
    +    // Entity::postSave() calls Entity::invalidateTagsOnSave(), which only
    +    // handles the regular cases. The Shortcut entity has one special case: a
    +    // newly created shortcut is *also* added to a shortcut set, so we must
    +    // invalidate the associated shortcut set's cache tag.
    +    if (!$update) {
    +      Cache::invalidateTags($this->getCacheTag());
    +    }
    

    Entity::invalidateTagsOnSave() also invalidates $this->getListCacheTags(), so if we're returning the correct tags in our getListCacheTags() implementation, why is this neccessary?

  4. +++ b/core/modules/shortcut/src/Entity/Shortcut.php
    @@ -199,4 +221,23 @@ public static function baseFieldDefinitions(EntityTypeInterface $entity_type) {
    +  public function getCacheTag() {
    +    $field_value = $this->get('shortcut_set')
    +      ->getValue(TRUE);
    +    $shortcut_set = $field_value[0]['entity'];
    +    return $shortcut_set->getCacheTag();
    +  }
    

    As said above, are we sure we need this?

  5. +++ b/core/modules/shortcut/src/Entity/Shortcut.php
    @@ -199,4 +221,23 @@ public static function baseFieldDefinitions(EntityTypeInterface $entity_type) {
    +    $field_value = $this->get('shortcut_set')->getValue(TRUE);
    +    $shortcut_set = $field_value[0]['entity'];
    +    return $shortcut_set->getListCacheTags();
    

    I think you mean:

    $this->shortcut_set->entity->getListCacheTags().

    Same for getCacheTag() above if we decide to keep it.

  6. +++ b/core/modules/shortcut/src/Tests/ShortcutCacheTagsTest.php
    @@ -0,0 +1,77 @@
    +    /** @var \Drupal\shortcut\Entity\Shortcut $shortcut */
    

    I don't think we need ugly inline comments like this if we're not even using any special method from the Shortcut class. And btw, it should hint the interface :)

wim leers’s picture

StatusFileSize
new18.42 KB
new2.93 KB

Thanks for the review! :)

  1. No, because cache tags are of the form array('entity type' => 'entity ID'). So $a = array('shortcut' => array(1)) and array('shortcut' => array(2)); $a + $b === array('shortcut' => array(1)) which is wrong, NestedArray::mergeDeep($a, $b) === array('shortcut' => array(1, 2)), which is right.
  2. Well-spotted, sir! The ->isNew() is unnecessary there. The other part is necessary though: $this->path is only set when an entity is created, not when an entity is loaded. Therefore, it's necessary to prevent this from fataling: $shortcut = entity_load('shortcut', 343); $shortcut->save();, which happens as part of \Drupal\system\Tests\Entity\EntityCacheTagsTestBase::testReferencedEntity().
    The proper solution would be to not have $this->path at all, but to always use routes in the Shortcut entity. I first considered that out of scope, but since you brought it up, I might as well fix it. Fixed.
  3. The code you cite does not have the list tag (array('shortcut_sets' => TRUE), but the entity's unique tag (array('shortcut_set' => '<id>'))! The entity's unique tag is invalidated only upon update by Entity::invalidateTagsOnSave(), but we need it to be invalidated upon creation as well: when a shortcut is created, it's always associated with a shortcut set, and since a shortcut uses the associated shortcut set's cache tags, we must invalidate that tag.
  4. Hm… this is something else entirely: this is how Shortcut tells Drupal to use the associated ShortcutSet's cache tags instead. I think you're confused? :)
  5. No, this is correct also. Shortcut::getCacheTag() returns ShortcutSet::getCacheTag(), and Shortcut::getListCacheTags() returns ShortcutSet::getListCacheTags(). Simple forwarding. I think you thought ::getListCacheTags() is a superset of ::getCacheTag(), but it's not.
  6. Haha, oops! I thought this was a supposed standard, but I don't care. Removed.

Status: Needs review » Needs work

The last submitted patch, 13: shortcut_cache_tags-2241235-13.patch, failed testing.

berdir’s picture

Also stumbled over the postCreate() problem in #2183231-260: Make ContentEntityDatabaseStorage generate static database schemas for content entities, a bit a different fix there without the weird isOriginalId(), see also the comment about @alexpott suggesting that the whole thing should live in the form submit or so.

wim leers’s picture

StatusFileSize
new17.13 KB
new2.79 KB
new4.68 KB

Clearly, I didn't test #13.2 sufficiently. In trying to fix it, I tried to understand the complex relationship that Shortcut has with a "path" field. Its "path" field is a computed field that is actually used for input to compute the actual values to be stored, which are then used to compute the path when editing again… I found a solution for that, but then I ran into problems with MapItem, which I suspect is buggy and may also be used incorrectly.
I spent more than three hours trying to get this to work, digging through the many layers of Entity/Field API (with many layers of "setting" and "getting" values) … and just when I was about to give up, I found #2235457: Use link field for shortcut entity, which seems to solve all of this :) A nice 3 hours wasted. I learned my lesson: I'll look for existing issues sooner next time. (In case it turns out to be useful after all: I've attached an "out of scope" interdiff.)

So this reroll:

All this slowdown is all due to out-of-scope issues. Let's focus on the issue at hand: make shortcuts use the cache tags of shortcut sets, and make sure they're invalidated correctly.

wim leers’s picture

Status: Needs work » Needs review
wim leers’s picture

StatusFileSize
new16.93 KB
new1.02 KB

Talked to amateescu in IRC, turns out I completely misunderstood what he meant in #12.4 & #12.5. See the interdiff for what he meant :)

Much cleaner now! :)

amateescu’s picture

Status: Needs review » Reviewed & tested by the community

Cool, this looks ready to go now.

catch’s picture

Status: Reviewed & tested by the community » Fixed

Committed/pushed to 8.x, thanks!

  • Commit a78a106 on 8.x by catch:
    Issue #2241235 by Wim Leers: Shortcut/ShortcutSet entity types should...
wim leers’s picture

moshe weitzman’s picture

wim leers’s picture

#23: no, that's completely independent of what this issue tackled.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.