Problem/Motivation

In contrast to D7 we do load a lot of files before page cache, even we don't actually need them.

Proposed resolution

Move the loads to preHandle and see how much we gain.

Remaining tasks

User interface changes

API changes

  • Remove logging from ControllerResolver.
  • Add method to DrupalKernel to load legacy includes.

Beta phase evaluation

Reference: https://www.drupal.org/core/beta-changes
Issue priority Major because it improves performance by 25% for cached pages
Prioritized changes The main goal of this issue is performance.
CommentFileSizeAuthor
#30 2392787-28.patch9.86 KBalexpott
#25 2392787-25.patch9.97 KBAnonymous (not verified)
#22 2392787-22-interdiff.txt694 bytesAnonymous (not verified)
#22 2392787-22.patch9.99 KBAnonymous (not verified)
#20 2392787-20-interdiff.txt1.28 KBAnonymous (not verified)
#20 2392787-20.patch9.51 KBAnonymous (not verified)
#15 2392787-15-interdiff.txt1.84 KBAnonymous (not verified)
#15 2392787-15.patch9.33 KBAnonymous (not verified)
#10 interdiff.txt547 bytesdawehner
#10 2392787-10.patch7.67 KBdawehner
#7 2392787-7.patch7.14 KBdawehner
#3 2392787-3.patch4.23 KBdawehner
#1 2392787-1.patch2.5 KBdawehner
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

dawehner’s picture

Status: Active » Needs review
FileSize
2.5 KB

So the result is the following:

Before

Time taken for tests:   26.357 seconds

After

Time taken for tests:   26.089 seconds

Status: Needs review » Needs work

The last submitted patch, 1: 2392787-1.patch, failed testing.

dawehner’s picture

Status: Needs work » Needs review
FileSize
4.23 KB

Let's move drupal_get_path() to bootstrap.inc to get around that.

Status: Needs review » Needs work

The last submitted patch, 3: 2392787-3.patch, failed testing.

Berdir’s picture

Some more numbers, based on my testing, a page cache response in HEAD is 4.2ms (with the page cache no config thing enabled), with #2392433: Stream wrappers are registered before page cache, the same change as here + the removal of the injected logger from ControllerResolver (this part will be part of the next patch), it is 3.1ms

With all those changes, we're executing 2 queries, for the page cache entry and the cache tags checks for it.

That's an improvement of 25%.

dawehner’s picture

Priority: Normal » Major
Status: Needs work » Needs review

Before

Requests per second:    39.46 [#/sec] (mean)
Time per request:       25.344 [ms] (mean)

After

Requests per second:    46.33 [#/sec] (mean)
Time per request:       21.586 [ms] (mean)
dawehner’s picture

FileSize
7.14 KB

Status: Needs review » Needs work

The last submitted patch, 7: 2392787-7.patch, failed testing.

Anonymous’s picture

i like the idea, seems like a simple change that should work.

if it doesn't, we should fix the things that stop it working.

dawehner’s picture

Status: Needs work » Needs review
Issue tags: +Ghent DA sprint
FileSize
7.67 KB
547 bytes

That was a simple fix, maybe someone has a cleaner solution though.

dawehner queued 10: 2392787-10.patch for re-testing.

Wim Leers’s picture

Status: Needs review » Reviewed & tested by the community

99% of this patch is trivial moving around of code. If testbot says it's green, and I can still install Drupal manually, then it's good to go.

The only potentially contentious change is this one:

+++ b/core/core.services.yml
@@ -400,7 +400,7 @@ services:
-    arguments: ['@class_resolver', '@logger.channel.default']
+    arguments: ['@class_resolver']

This is okay because it is only necessary for this code in \Symfony\Component\HttpKernel\Controller\ControllerResolver:

    public function getController(Request $request)
    {
        if (!$controller = $request->attributes->get('_controller')) {
            if (null !== $this->logger) {
                $this->logger->warning('Unable to look for the controller as the "_controller" parameter is missing');
            }

… which is done only for DX, but that in fact also happens in HttpKernel::handleRaw():

        // load controller
        if (false === $controller = $this->resolver->getController($request)) {
            throw new NotFoundHttpException(sprintf('Unable to find the controller for path "%s". The route is wrongly configured.', $request->getPathInfo()));
        }

So if this costs us performance, and we have an equivalent to maintain a good DX, this doesn't actually change anything.

Berdir’s picture

Should we remove the logging code in ControllerResolver completely? If we never inject the logger, then keeping that code is a bit pointless, especially as we already deal with it one layer above?

Wim Leers’s picture

Status: Reviewed & tested by the community » Needs work

That's a good point. Minor, but good. Let's do that.

Anonymous’s picture

Status: Needs work » Needs review
FileSize
9.33 KB
1.84 KB

like this?

Wim Leers’s picture

Status: Needs review » Reviewed & tested by the community

Yes, exactly like that. Thanks!

alexpott’s picture

Status: Reviewed & tested by the community » Needs review
+++ b/core/lib/Drupal/Core/DrupalKernel.php
@@ -436,6 +419,22 @@ public function getContainer() {
   public function preHandle(Request $request) {
+    require_once $this->root . '/core/includes/common.inc';
+    require_once $this->root . '/core/includes/database.inc';
+    require_once $this->root . '/core/includes/path.inc';
+    require_once $this->root . '/core/includes/module.inc';
+    require_once $this->root . '/core/includes/theme.inc';
+    require_once $this->root . '/core/includes/pager.inc';
+    require_once $this->root . '/core/includes/menu.inc';
+    require_once $this->root . '/core/includes/tablesort.inc';
+    require_once $this->root . '/core/includes/file.inc';
+    require_once $this->root . '/core/includes/unicode.inc';
+    require_once $this->root . '/core/includes/form.inc';
+    require_once $this->root . '/core/includes/mail.inc';
+    require_once $this->root . '/core/includes/errors.inc';
+    require_once $this->root . '/core/includes/schema.inc';
+    require_once $this->root . '/core/includes/entity.inc';
+

I'm wondering if we should move this into a method so that if we ever get to the point that Drush does not need to fake a request it can do something cleanly. Tying all this includes into the internal implementation of preHandle feels wrong.

Fabianx’s picture

RTBC +1 once #17 is addressed, a helper function makes a lot of sense.

Anonymous’s picture

ok, so what should we call the method? loadLegacyCode() ? loadOldDrupalCodeOhHowIMissThee() ?

Anonymous’s picture

ok, i chose loadLegacyIncludes().

dawehner’s picture

Meh, I guess we have to add the method to the DrupalKernelInterface?

Anonymous’s picture

FileSize
9.99 KB
694 bytes

mkay then.

Wim Leers’s picture

+++ b/core/lib/Drupal/Core/DrupalKernel.php
@@ -433,9 +416,33 @@ public function getContainer() {
+   * Load legacy include files.

This should now become {@inheritdoc}.

tstoeckler’s picture

Re #17: Not objecting, but it is still the case that (among others) the UrlGenerator expects a RequestContext and just fatals if it's not set, so I think Drush (and any other external scripts) not having to call preHandle() is a pipe dream at this point. We just have to accept the fact that preHandle() is the equivalent of DRUPAL_BOOTSTRAP_FULL now...

Anonymous’s picture

FileSize
9.97 KB

mkay, added inheritdoc.

dawehner’s picture

Status: Needs review » Reviewed & tested by the community

+1

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 25: 2392787-25.patch, failed testing.

dawehner’s picture

Status: Needs work » Reviewed & tested by the community
FileSize
9.86 KB

Just a usual reroll.

alexpott’s picture

Issue summary: View changes

Updated summary.

alexpott’s picture

FileSize
9.86 KB

xpost that somehow deleted @dawehner's patch :(

webchick’s picture

Assigned: Unassigned » catch

Wow, according to the issue summary, this sounds great! :) Moving over to catch for sign-off since it deals with bootstrap performance.

catch’s picture

Status: Reviewed & tested by the community » Fixed

This looks good, although loading those files on page cache requests is a bad regression from Drupal 7 so it's sadly not an improvement as such.

This doesn't need profiling, just common sense.

Committed/pushed to 8.0.x, thanks!

  • catch committed 7d3be89 on 8.0.x
    Issue #2392787 by beejeebus, dawehner, alexpott: Move include statements...
Wim Leers’s picture

Note: see #1 and #6 for profiling. This was also profiled as part of #2392433: Stream wrappers are registered before page cache, which is where this problem was discovered IIRC.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.