Problem/Motivation

THIS IS NOT ABOUT CACHING THE ACTUAL ROUTING

In order to be able to support linking to paths directly and checking access, we need to do
the following:

  • Convert the path to a route on runtime
  • Check access using that route

The second bit will be solved by cacheable access checking.
The first bit is not cacheable first, let's do that.

Proposed resolution

Given that the determined route of a menu link just varies by path, we can use the UrlMatcherInterface and provide a wrapper
which caches the result. With that checking access to path based menu links can be done faster in the future.

Ones we have the path => route resolution cached, we can store the paths for menu links, not having to worry about the access checking performance
in the future.

Even if core doesn't implement it, also contrib could implement a path only based menu link and reuse the functionality added in this issue.

Remaining tasks

User interface changes

API changes

Beta phase evaluation

Reference: https://www.drupal.org/core/beta-changes
Issue category Tasks because it allows us to to use paths potentially for menu links without a big sacrifice of performance
Issue priority TODO
Disruption Not disruptived at all, it just adds an internal API.
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

dawehner’s picture

Status: Active » Needs review
FileSize
5.46 KB

There we go.

Status: Needs review » Needs work

The last submitted patch, 1: 2370651-1.patch, failed testing.

dawehner’s picture

Status: Needs work » Needs review
FileSize
5.54 KB
1.53 KB

Ha.

Status: Needs review » Needs work

The last submitted patch, 3: 2370651-3.patch, failed testing.

dawehner’s picture

Status: Needs work » Needs review
Issue tags: +Random test failure
FileSize
5.54 KB

This seems to be a random one.

Status: Needs review » Needs work

The last submitted patch, 5: 2370651-3.patch, failed testing.

dawehner’s picture

Status: Needs work » Needs review
FileSize
7.16 KB

... Added some debug statements.

Status: Needs review » Needs work

The last submitted patch, 7: 2370651-7.patch, failed testing.

amateescu’s picture

dawehner’s picture

Seems not a random test failure but I cannot really preproduce it.

dawehner’s picture

Issue summary: View changes

Status: Needs work » Needs review

dawehner queued 7: 2370651-7.patch for re-testing.

Status: Needs review » Needs work

The last submitted patch, 7: 2370651-7.patch, failed testing.

dawehner’s picture

Issue summary: View changes
Status: Needs work » Needs review
FileSize
6.27 KB
1.85 KB

Ha!

jibran’s picture

Status: Needs review » Needs work
Issue tags: -Random test failure +Need tests

I believe we need tests here.

dawehner’s picture

Status: Needs work » Needs review
Issue tags: -Need tests
FileSize
13.25 KB
7.6 KB

Alright, this time with more test coverage.

catch’s picture

Priority: Normal » Major

See #643984: Cache menu_get_item() and #1234830: Revert cache_menu patch removal, add a $conf setting instead (was: cache_menu: huge table size).

Short version - this is an absolute necessity to keep some sites up (MySQL query cache saved by memcache), but it can also take other sites down (cache table size growing out of control using MySQL for the cache).

On the 7.x issue we were discussing making it configurable.

For 8.x I'm wondering if we want a special cache bin that requires a size-limited cache backend, but otherwise uses the null or memory cache backend if one isn't available.

dawehner’s picture

Mh, this is not yet caching the routing by request.

@catch
Was this caused by the menu_get_item() call for routing or by the ones needed for menu item loading?

catch’s picture

Menu items never had a menu_get_item() to load the route - the route was loaded with the menu links as part of the database query via join. This is part of the reason Drupal 8 routing is so much slower for menu link rendering. See for example #1845402-5: Update menu link access checks for new router for a slightly longer description of the regression.

So while there can be other menu_get_item() calls with a path name in 7.x, nearly all of them were for the incoming request not for link generation.

The issue here is that there can be potentially millions of valid paths on a site (nodes + users + $entity_type), and if the site is being crawled that will fill the cache quite quickly. Also the original patch that went in had this as a permanent cache entry (since it's valid until the router changes).

dawehner’s picture

FileSize
24.73 KB
11.93 KB
  • Decided to move the cache documentation into settings.php. Feel free to disagree but I think the visibility is way more visible there.
  • Used cache.menu instead of cache.data for the CachedUrlMatcher
  • Added some basic caching for routing itself.

Status: Needs review » Needs work

The last submitted patch, 20: 2370651-20.patch, failed testing.

dawehner’s picture

Status: Needs work » Needs review
FileSize
562 bytes
24.74 KB

Ups.

dawehner’s picture

FileSize
25.22 KB
495 bytes

chx made a point.

Fabianx’s picture

I would totally RTBC this, it looks great!

alexpott’s picture

Status: Needs review » Needs work
  1. +++ b/core/core.services.yml
    @@ -520,6 +522,12 @@ services:
    +  router.cached:
    +    class: Drupal\Core\Routing\CachedUrlMatcher
    

    The service or class name should be more consistent?

  2. +++ b/core/lib/Drupal/Core/Access/AccessManager.php
    @@ -221,5 +229,19 @@ protected function performCheck($service_id, ArgumentsResolverInterface $argumen
    +  public function setUrlMatcher(UrlMatcherInterface $url_matcher) {
    +    $this->urlMatcher = $url_matcher;
    +
    +    return $this;
    +  }
    

    Missing documentation - looks like it should be on the interface.

  3. It would be great if the issue summary could be updated to have the plan going forward as this patch adds lots of code that is unused atm
  4. +++ b/core/lib/Drupal/Core/Routing/RouteProvider.php
    @@ -71,10 +79,11 @@ class RouteProvider implements RouteProviderInterface, PagedRouteProviderInterfa
    +  public function __construct(Connection $connection, RouteBuilderInterface $route_builder, StateInterface $state, CacheBackendInterface $cache_backend, $table = 'router') {
    
    @@ -276,6 +285,9 @@ public function getRoutesByPattern($pattern) {
    +    if ($cache = $this->cacheBackend->get('routing_by_path:' . $path)) {
    +      return $cache->data;
    +    }
    
    @@ -301,6 +313,8 @@ protected function getRoutesByPath($path) {
    +    $this->cacheBackend->set('routing_by_path:' . $path, $collection);
    

    Currently this will result in just shoving stuff in memory that is never used. How caching if a cache backend is injected and defaulting to the backend to NULL.

alexpott’s picture

dawehner’s picture

Issue summary: View changes
Status: Needs work » Needs review
Related issues: +#2406117: potentially cache parts of routing
FileSize
8.58 KB
19.29 KB

Missing documentation - looks like it should be on the interface.

I on purpose did not set it onto the interface, because its a pure implementation detail to avoid circular dependencies.

It would be great if the issue summary could be updated to have the plan going forward as this patch adds lots of code that is unused atm

I hope this new paragraph helps a bit.

Currently this will result in just shoving stuff in memory that is never used. How caching if a cache backend is injected and defaulting to the backend to NULL.

@catch request caching of those bits, but I don't want to introduce it at that point, given that its a more complex topic than path only based routing.
Let's split up this part into another issue: #2406117: potentially cache parts of routing

Status: Needs review » Needs work

The last submitted patch, 27: 2370651-27.patch, failed testing.

dawehner’s picture

Status: Needs work » Needs review
FileSize
31.95 KB
2.36 KB

Fixed that.

Status: Needs review » Needs work

The last submitted patch, 29: 2370651-29.patch, failed testing.

dawehner’s picture

Status: Needs work » Needs review
FileSize
16.93 KB
15.02 KB

This contained some unrelated changes from a different patch

jibran’s picture

BE in IS is not complete and one minor nit. I think it is ready.

+++ b/sites/default/default.settings.php
@@ -390,6 +390,30 @@
+// Per default cache the menu and routing just in memory. Given that there a lot
+// of possible allowed paths you should better use a cache backend, which has
+// a limited total size / limited amount of items. Database cache would
+// potentially fill up a lot.

I think line commenting is wrong here we should use block commenting.

dawehner’s picture

FileSize
16.96 KB
905 bytes

Rerolled and fixed the points in the review #32.

Status: Needs review » Needs work

The last submitted patch, 33: 2370651-33.patch, failed testing.

dawehner’s picture

Issue summary: View changes
dawehner’s picture

Title: Make routing by path cacheable » Make (routing by path) cacheable

Adapt the title.

dawehner’s picture

kim.pepper’s picture

Status: Needs work » Needs review
FileSize
15.45 KB
597 bytes

Fixes editable config test failure.

Status: Needs review » Needs work

The last submitted patch, 38: 2370651-path-route-cache-38.patch, failed testing.

kim.pepper’s picture

Status: Needs work » Needs review
FileSize
15.21 KB
818 bytes

Fix more editable config fails.

aspilicious’s picture

Status: Needs review » Needs work

A file has been deleted that shouldn't be deleted ;)

kim.pepper’s picture

Status: Needs work » Needs review
FileSize
15.18 KB
444 bytes

Whoops!

bill richardson’s picture

Status: Needs review » Needs work

Patch requires re-roll - setting to needs work.

dawehner’s picture

Status: Needs work » Closed (won't fix)

Yeah I don't think we will need this any longer