Problem/Motivation

Follow-up for #2429617: Make D8 2x as fast: Dynamic Page Cache: context-dependent page caching (for *all* users!).

See #219 & #276.4 + replies.

Proposed resolution

  1. Don't vary by route, but by url, this allows us to avoid routing altogether.
  2. Somehow make sure we don't need to run the authentication subscriber anymore, so that we don't need to boot Drupal in order to know the values for the user.* cache contexts. (See #7 for a potential idea.)

Remaining tasks

TBD.

User interface changes

None.

API changes

None.

Data model changes

None.

Comments

fabianx’s picture

I am all for making things work with url instead of [route] and rely on the cacheable metadata added during access handling on the request object and we can cache the user related cache contexts per 'user session'.

That would then be the same strategy as my ESI v2 battleplan takes, just inside of core instead of in Varnish.

One caveat to all of that is:

- I had problems rendering placeholders without http kernel having run first, because something broke.

So likely this pre-routing middleware is only useful if there are no placeholders that are not cached / cacheable.

wim leers’s picture

Status: Postponed » Active
almaudoh’s picture

Title: Investigate moving SmartCache into a middleware at the cost of doing authentication manually if needed » Investigate moving Dynamic Page Cache into a middleware at the cost of doing authentication manually if needed
effulgentsia’s picture

Priority: Normal » Major
Issue tags: +Performance

Raising priority because I suspect the Performance gain from this will be quite large.

effulgentsia’s picture

Version: 8.0.x-dev » 8.1.x-dev

Moving to 8.1 though.

I had problems rendering placeholders without http kernel having run first, because something broke.

If this is still the case, then perhaps this issue could be done in two steps? First step to move it to a much higher priority REQUEST listener to at least factor out dependencies on routing and route-based authentication. Then a second step to move it to a middleware to factor out dependencies on http kernel.

fabianx’s picture

Yes, agree with #5 to do this in two steps.

wim leers’s picture

Issue summary: View changes

Also, as noted in http://wimleers.com/article/drupal-8-dynamic-page-cache#authcache — Authcache uses an interesting trick to sidestep the vast majority of the cost. I'm not sure how safe it is nor if it works with every authentication mechanism. I'm certain we can't use it for Dynamic Page Cache directly, because it relies on every response of the site varying by the same cache contexts (which is grossly inefficient).

But I think we should consider learning from it.

  • The first request from a certain session causes us to bootstrap as far as we do today in HEAD. We store the values for all user.* cache contexts in a cache entry: roles, permissions hash, UID, etc.
  • On subsequent cache requests, we then no longer need to boot very far anymore to do authentication.

(IS updated.)

fabianx’s picture

#7: Yes, that is the same authentication step per session cookie that I am doing in ESI.

In essence caching the cache contexts.

wim leers’s picture

In essence caching the cache contexts.

Yep, and important to note that we cache that per session.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.0-beta1 was released on March 2, 2016, which means new developments and disruptive changes should now be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.0-beta1 was released on August 3, 2016, which means new developments and disruptive changes should now be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

wim leers’s picture

Component: request processing system » dynamic_page_cache.module

I just noticed this is in the wrong component. Probably the right one didn't exist yet at the time.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

smustgrave’s picture

Status: Active » Postponed (maintainer needs more info)
Issue tags: +stale-issue-cleanup

Thank you for creating this issue to improve Drupal.

We are working to decide if this task is still relevant to a currently supported version of Drupal. There hasn't been any discussion here for over 8 years which suggests that this has either been implemented or is no longer relevant. Your thoughts on this will allow a decision to be made.

Since we need more information to move forward with this issue, the status is now Postponed (maintainer needs more info). If we don't receive additional information to help with the issue, it may be closed after three months.

Thanks!

catch’s picture

Per route significantly improves things for 404s because the page rendering itself all happens on the same route (e.g. the custom 404 page) (see #3516169: Adding url.path to breadcrumb cache context results in every 404 page getting a different dynamic page cache entry for how that improvement can then be undone) so I'm not sure this would necessarily be an improvement across the lifecycle of a site.

berdir’s picture

If anything, it should only move to before routing. We do a lot of post processing on dynamic page hits, such as building uncacheable or cache miss elements like blocks. This can not and will never work in a middleware without loaded modules and so on.

Whether that's feasible at all or not I don't know, didn't think through the 404 case at all. Also BC and implications on existing code that needs to run before/after seems very complicated. I'd say won't fix.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.