Problem/Motivation

Currently, the menu tree parameters are cached per menu, per route match at \Drupal\Core\Menu\MenuLinkTree::getCurrentRouteMenuTreeParameters().
This means that:

  • we do a cache get per menu that gets rendered
  • but we don't ever call that method when the menu block is render cached
  • and the menu block is currently already render cached (despite that caching actually being wrong/broken, but that's out of scope; when #1805054: Cache localized, access filtered, URL resolved, and rendered menu trees lands, menu blocks will still be render cached, so this analysis remains valid)

But, worse:

  • to retrieve render cached menu blocks, we need to calculate cache IDs
  • the cache IDs depend on active menu trails
  • active menu trails are currently not cached at all
  • which means that if we have N menus on a page, we also perform N queries
  • (Standard profile has 4: account, main, tools and footer)

Proposed resolution

It'd be better to remove the barely useful caching in MenuLinkTree::getCurrentRouteMenuTreeParameters(), because that is only used on cache misses, and instead apply caching to the active trails, which happens is calculated for every menu on the page.

That replaces N DB queries with a single cache get.

Remaining tasks

Review.

User interface changes

None.

API changes

None.

Comments

wim leers’s picture

Status: Active » Needs review
StatusFileSize
new8.7 KB

Benchmarking

Always 1000 requests.

Front page, UID 2

Before
Requests per second:    12.40 [#/sec] (mean)
Time per request:       80.620 [ms] (mean)
Time per request:       80.620 [ms] (mean, across all concurrent requests)
Transfer rate:          147.31 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:    76   80   1.5     80      87
Waiting:       68   71   1.4     71      78
Total:         77   81   1.5     80      87

After
Requests per second:    12.58 [#/sec] (mean)
Time per request:       79.485 [ms] (mean)
Time per request:       79.485 [ms] (mean, across all concurrent requests)
Transfer rate:          156.88 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:    75   79   1.5     79      90
Waiting:       66   70   1.4     70      78
Total:         75   79   1.5     79      90

Conclusion
~1ms faster

Front page, UID 1

Before
Requests per second:    9.15 [#/sec] (mean)
Time per request:       109.239 [ms] (mean)
Time per request:       109.239 [ms] (mean, across all concurrent requests)
Transfer rate:          175.00 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:   104  109   1.7    109     116
Waiting:       94   98   1.7     98     104
Total:        104  109   1.7    109     116
After
Requests per second:    9.27 [#/sec] (mean)
Time per request:       107.853 [ms] (mean)
Time per request:       107.853 [ms] (mean, across all concurrent requests)
Transfer rate:          182.71 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:   103  108   1.8    107     117
Waiting:       92   97   1.8     96     106
Total:        103  108   1.8    108     117
Conclusion
~1ms faster

Profiling

Function calls, UID 1, frontpage

Before
65302
After
64370
Conclusion
Minus 932, 1.4% less.

Status: Needs review » Needs work

The last submitted patch, 1: menu_active_trail_caching-2479363-1.patch, failed testing.

wim leers’s picture

Issue summary: View changes
StatusFileSize
new3.85 KB
new4.03 KB

I forgot to attach the histograms I made.

wim leers’s picture

Status: Needs work » Needs review
StatusFileSize
new10.96 KB
new7.13 KB

#1 came directly from old work on #1805054: Cache localized, access filtered, URL resolved, and rendered menu trees that had been lingering around. I just dusted it off and got it to work again. I should've run tests too, obviously.

I fixed this by adopting the Cache Collector pattern. Performance numbers are the same. Just 8 function calls less.

dawehner’s picture

Issue summary: View changes

Quick fix of the issue summary to not contain the same picture twice

wim leers’s picture

Issue summary: View changes

Oops, didn't mean to put the images in the IS at all! Dreditor--.

catch’s picture

Noticed today that apart from actual router matching, this is the only non-cache, non-state database query we execute in the minimal profile on every request.

So it represents potentially a 50% reduction in queries to the actual database when using different cache/state backends.

dawehner’s picture

I'm curious, how often do we things practically store per URL already?

+++ b/core/lib/Drupal/Core/Menu/MenuActiveTrail.php
@@ -38,16 +41,66 @@ class MenuActiveTrail implements MenuActiveTrailInterface {
+    parent::__construct(NULL, $cache, $lock);

Should we document that CID will be calculated in getCid() ?

  1. +++ b/core/lib/Drupal/Core/Menu/MenuActiveTrail.php
    @@ -38,16 +41,66 @@ class MenuActiveTrail implements MenuActiveTrailInterface {
    +      ksort($route_parameters);
    +      return 'active-trail:route:' . $this->routeMatch->getRouteName() . ':route_parameters:' . serialize($route_parameters);
    +    }
    

    I like that a lot that this is all still inside MenuActiveTrail, so in case someone decided to build a trail, for example based upon history, it would be possible to do so.

  2. +++ b/core/lib/Drupal/Core/Menu/MenuLinkTree.php
    @@ -84,53 +60,31 @@ class MenuLinkTree implements MenuLinkTreeInterface {
       public function getCurrentRouteMenuTreeParameters($menu_name) {
    -    $route_parameters = $this->routeMatch->getRawParameters()->all();
    -    ksort($route_parameters);
    -    $cid = 'current-route-parameters:' . $menu_name . ':route:' . $this->routeMatch->getRouteName() . ':route_parameters:' . serialize($route_parameters);
    -
    -    if (!isset($this->cachedCurrentRouteParameters[$cid])) {
    -      $cache = $this->cache->get($cid);
    -      if ($cache && $cache->data) {
    -        $parameters = $cache->data;
    -      }
    -      else {
    -        $active_trail = $this->menuActiveTrail->getActiveTrailIds($menu_name);
    -
    -        $parameters = new MenuTreeParameters();
    -        $parameters->setActiveTrail($active_trail)
    -          // We want links in the active trail to be expanded.
    -          ->addExpandedParents($active_trail)
    -          // We marked the links in the active trail to be expanded, but we also
    -          // want their descendants that have the "expanded" flag enabled to be
    -          // expanded.
    -          ->addExpandedParents($this->treeStorage->getExpanded($menu_name, $active_trail));
    -
    -        $this->cache->set($cid, $parameters, CacheBackendInterface::CACHE_PERMANENT, array('config:system.menu.' . $menu_name));
    -      }
    -      $this->cachedCurrentRouteParameters[$menu_name] = $parameters;
    -    }
    -
    -    return $this->cachedCurrentRouteParameters[$menu_name];
    +    $active_trail = $this->menuActiveTrail->getActiveTrailIds($menu_name);
    

    Great change!

  3. +++ b/core/modules/system/src/Tests/Block/SystemMenuBlockTest.php
    index 2ea72ff..16f4b8c 100644
    --- a/core/tests/Drupal/Tests/Core/Menu/MenuActiveTrailTest.php
    
    --- a/core/tests/Drupal/Tests/Core/Menu/MenuActiveTrailTest.php
    +++ b/core/tests/Drupal/Tests/Core/Menu/MenuActiveTrailTest.php
    
    +++ b/core/tests/Drupal/Tests/Core/Menu/MenuActiveTrailTest.php
    @@ -62,8 +62,10 @@ protected function setUp() {
    +    $cache = $this->getMock('\Drupal\Core\Cache\CacheBackendInterface');
    +    $lock = $this->getMock('\Drupal\Core\Lock\LockBackendInterface');
     
    -    $this->menuActiveTrail = new MenuActiveTrail($this->menuLinkManager, $this->currentRouteMatch);
    +    $this->menuActiveTrail = new MenuActiveTrail($this->menuLinkManager, $this->currentRouteMatch, $cache, $lock);
       }
    

    I'm curious whether we should adapt the unit test to test the new caching code, it would be possible, this is for sure.

wim leers’s picture

I'm curious, how often do we things practically store per URL already?

In this case, we already were caching similar data per URL, but now we're just being smarter about that. See the IS for details. :)

Should we document that CID will be calculated in getCid() ?

No, because this is a pattern that you see elsewhere too, e.g. in \Drupal\Core\Asset\LibraryDiscoveryCollector.

  1. Exactly :) Self-contained.
  2. :)
  3. IMHO this is not very valuable. We just adopt a well-established/tested caching pattern. Testing that the caching (well, its invalidation) itself is working correctly is IMHO a waste of time, since we already have high confidence that the building blocks that we're reusing are working correctly.
catch’s picture

I'm curious, how often do we things practically store per URL already?

Path alias pre-load cache is also per-URL.

dawehner’s picture

Well, I'd kinda like to save us from breaking the following piece of code for example:

+  protected function getCid() {
+    if (!isset($this->cid)) {
+      $route_parameters = $this->routeMatch->getRawParameters()->all();
+      ksort($route_parameters);
+      return 'active-trail:route:' . $this->routeMatch->getRouteName() . ':route_parameters:' . serialize($route_parameters);
+    }

Nothing documents why the ksort() is needed, yes I know, but maybe someone else doesn't know in the future. Having test coverage for that helps against that.

dawehner’s picture

StatusFileSize
new14.43 KB
new4.92 KB

Well, then lets do it. I think the patch itself is sane

wim leers’s picture

+++ b/core/tests/Drupal/Tests/Core/Menu/MenuActiveTrailTest.php
@@ -163,7 +184,54 @@ public function testGetActiveTrailIds(Request $request, $links, $menu_name, $exp
+      ->with('baby_llama')

I approve of this test!

Thanks for the test coverage!

(I was thinking of using the 'route' cache context instead; but that'd have required me to inject CacheContextsManager here, which would've been ugly. This is better.)

fabianx’s picture

Status: Needs review » Reviewed & tested by the community

RTBC, looks great now with the added test coverage.

Train of thought

I pondered this change for a moment, but because blocks are plugins it is pretty unlikely that someone will display a menu manually without going through the block (or some other #pre_render cache).

[ Also the code to display a menu manually is really complicated and placing a block is really simple. ]

catch’s picture

Status: Reviewed & tested by the community » Fixed

This looks great. I also saw these queries when profiling with the minimal profile/front page/logged out - they're the only uncached database queries apart from path alias and routing lookups. So this will both reduce the number of queries and it also cuts out overhead from SelectQuery.

Committed/pushed to 8.0.x, thanks!

  • catch committed e8a5d9e on 8.0.x
    Issue #2479363 by Wim Leers, dawehner: Cache MenuActiveTrail::...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.