Problem/Motivation

The UI installer rebuilds the dependency container on every request and sometimes more than once during a request when modules or themes are installed.

For tests and probably the CLI, this is fine, because there is only one 'request' and then the rest of the install happens inside it. For the UI installer, there are pages like the site settings form and translation import that don't trigger a container rebuild, and these are unnecessarily slow. A container rebuild can easily take hundreds of milliseconds.

Steps to reproduce

Proposed resolution

  • Allow dumping only once base_system_verified is TRUE (the prod-env phase, where nearly all requests happen: module install batches, recipes, translation import). The early installer may have no database and keeps compiling per request, so the settings-form/requirements pages are not covered by this.
  • InstallerKernel::getContainerCacheKey() gets an :installer suffix so the dumped installer container (with its service overrides) can never be served to the installed site.
  • install_drupal() deletes the dumped installer container via invalidateContainer() once installation_finished is set, after all tasks.

Three things had to be fixed to make dumping possible:

  1. NormalInstallerServiceProvider passes new Time() as a cache_factory argument — the dump throws "Unable to dump a service container if a parameter is an object". Fixed with new Reference('datetime.time').
  2. The same provider removes InlineServiceDefinitionsPass, RemoveUnusedDefinitionsPass, AnalyzeServiceReferencesPass and ReplaceAliasByActualDefinitionPass — safe only while never dumping; the dumped container otherwise breaks on reload with unresolved aliases/private services. Now conditional on the dumping phase.
  3. InstallerKernel::initializeContainer() forces containerNeedsRebuild = TRUE on every fresh boot, bypassing the cached-container read — reworked so a fresh boot consults the dump, keeping the in-request memoization and changed-extensions rebuild.

Performance results

All results were on a macOS host, DDEV v1.25.3 (Docker/Rancher Desktop), all executed inside the DDEV web container. Drupal CMS 2.x with core 11.4.5 with MariaDB. Core was main at 7d8143e6 vs the MR commit at 86d12f92 with PHP 8.5.7's built-in server with SQLite. Instrumentation was one temporary probe in DrupalKernel::initializeContainer(). Run with PHP submitting forms immediately (not through browser, so rendering, JS, etc. did not matter).

Ran 3 runs each of the before/after scenarios and averaged with variance included in the table.

Drupal CMS (Blank template) — Apache/MariaDB via DDEV, core 11.4.5 + adapted patch

Compiles Compile total Boots from dump Wall (start→finish)
EN before 38/40/38 6.53 ± 0.78 s 0 28.2 ± 3.7 s
EN after 29/29/29 4.73 ± 0.53 s 9/11/9 24.8 ± 1.9 s
DE before 53/54/53 12.94 ± 0.97 s 0 47.6 ± 3.2 s
DE after 30/30/30 4.94 ± 0.06 s 23/22/22 37.5 ± 0.5 s
EN delta −9 −1.8 s (−28 %) −3.4 s (−12 %)
DE delta −23 −8.0 s (−62 %) −10.0 s (−21 %)

Core standard profile — actual main (12.0-dev) vs MR commit 86d12f92, PHP 8.5/SQLite

Compiles Compile total Boots from dump Wall (start→finish)
EN before 11/11/11 0.36 ± 0.09 s 0 1.54 ± 0.29 s
EN after 7/7/7 0.29 ± 0.11 s 4/4/4 1.51 ± 0.37 s
DE before 19/19/19 0.66 ± 0.02 s 0 3.22 ± 0.09 s
DE after 9/9/9 0.33 ± 0.01 s 10/10/10 2.88 ± 0.09 s
EN delta −4 −0.07 s (−19 %) ~0
DE delta −10 −0.33 s (−50 %) −0.34 s (−11 %)

Variance in boots from dump are because the installer bundles batch requests in 1s chunks, so if the run happened to fit more into that, there would be less boots.

Remaining tasks

Review! Test!

User interface changes

None.

Introduced terminology

None.

API changes

None.

Data model changes

None.

Release notes snippet

The late installer now dumps the container to database once that is available, which results a roughly 10% server side speed improvement for foreign language Drupal core installations. Drupal CMS gets even bigger gains with roughly 10% server side speed improvement on English installs and 20% for foreign language.

LLM disclosure

LLM was used to build and measure this fix. The change is really small though and should be independently verified :)

Issue fork drupal-3619614

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

catch created an issue. See original summary.

gábor hojtsy made their first commit to this issue’s fork.

gábor hojtsy’s picture

I prototyped this with the help of an LLM and it appears to work wonders.

  • Allow dumping only once base_system_verified is TRUE (the prod-env phase, where nearly all requests happen: module install batches, recipes, translation import). The early installer may have no database and keeps compiling per request, so the settings-form/requirements pages from the issue summary are not covered by this.
  • InstallerKernel::getContainerCacheKey() gets an :installer suffix so the dumped installer container (with its service overrides) can never be served to the installed site.
  • install_drupal() deletes the dumped installer container via invalidateContainer() once installation_finished is set, after all tasks.

Three things had to be fixed to make dumping possible:

  1. NormalInstallerServiceProvider passes new Time() as a cache_factory argument — the dump throws "Unable to dump a service container if a parameter is an object". Fixed with new Reference('datetime.time').
  2. The same provider removes InlineServiceDefinitionsPass, RemoveUnusedDefinitionsPass, AnalyzeServiceReferencesPass and ReplaceAliasByActualDefinitionPass — safe only while never dumping; the dumped container otherwise breaks on reload with unresolved aliases/private services. Now conditional on the dumping phase.
  3. InstallerKernel::initializeContainer() forces containerNeedsRebuild = TRUE on every fresh boot, bypassing the cached-container read — reworked so a fresh boot consults the dump, keeping the in-request memoization and changed-extensions rebuild.

gábor hojtsy’s picture

Issue summary: View changes
gábor hojtsy’s picture

Status: Active » Needs review
gábor hojtsy’s picture

Issue summary: View changes
gábor hojtsy’s picture

I did a better Drupal CMS run averaging across 3 runs each (12 runs total), the English saves time in compilation but its not as big as with German. Will post in issue summary.

gábor hojtsy’s picture

Issue summary: View changes

Did another pass with 3 times running each before and after scenario through core UI installer on origin/main too and while compile time comes down, it is a wash overall in the final results. Updating issue summary.

gábor hojtsy’s picture

Issue summary: View changes

Updated issue summary with numbers :)

gábor hojtsy’s picture

Issue summary: View changes

Fix formatting.

gábor hojtsy’s picture

Issue summary: View changes
catch’s picture

If it's a wash for core and an improvement for Drupal CMS, that seems absolutely fine. We might also see different results with a core install from recipe or config directory etc.

I think we should double check that this isn't a performance regression for non-interactive installs, drush + dr + functional tests. I've been profiling functional tests recently and could check against those easily enough, although not today. Interactive install vs. not is in install state so we could probably vary on that if it's a problem.

nicxvan’s picture

I kind of want to do a drupal 10 and main comparison with all these improvements.

gábor hojtsy’s picture

Implemented @catch's suggestion of YAML re-parsing also not happening on each request. That helped with 2% better performance on core installed in German and 3% with CMS installed in German. The first page load after the installer is now also faster 10% or so compared to prior to the MR to after the current updated MR (even for English core install, but same relative improvement for German CMS install). Averaged results from 3 runs each.

andypost’s picture

has colisions in codebase as NormalInstallerServiceProvider and InstallerKernel::initializeContainer()