Problem/Motivation
Prior to #3571994: Reduce container rebuilds in functional tests we were rebuilding the container 11 times in functional tests. That issue got it down to approx. 8.
When profiling a couple of tests recently, I noticed that container rebuilds were still taking at least 50% of the time in a functional test that otherwise does very little, in the region of 8-14s give or take xhprof overhead.
Steps to reproduce
Proposed resolution
Some of these issues are specific to functional and functional js tests, some will also be reductions in UI or cli installs of real sites.
#3614153: Install system module alongside other modules in the installer
#3614401: Allow install profiles to opt out of being installed
#3614825: Allow additional modules to be installed alongside the profile modules
#3615878: [pp-1] Don't bother rebuilding the container after installing themes in the installer
#3574200: Get rid of 2-phase kernel boot in installer
#3615423: Avoid rebuilding the container in tests just to create a directory
#3615455: Install database driver modules alongside other modules
Bonus issues
Other installer/test performance improvements found while investigating this:
#3616275: Use the PECL YAML parser for info file parsing when available
#3615984: Set ::strictConfigSchema on a test by test basis in functional tests
Comments
Comment #2
catchComment #3
catchComment #4
catchComment #5
catchJust rebased #3574200: Get rid of 2-phase kernel boot in installer and stacked it on top of the other MRs in this issue, and with it we're down to three container builds in a minimal functional test.
- to build the initial installer container that allows us to write config etc.
- one when installing modules
- to get a production DrupalKernel after the Drupal install and prior to running tests.
This means that a real install via the cli is also down to a theoretical minimum of two container builds, although that relies on the install profile opting itself out.
There might be a way to optimize away the first one, not sure it is worth taking on until all the other issues have landed though and it would probably depend on #3583505: Use Symfony PhpDumper instead of a serialized array container structure too.
I think that we could potentially compile the early installer container to disk instead of using container builder, and commit it to core. This would require a core development script to rebuild the dumped container when we need to change anything with it, but that should happen very rarely. As soon as we install modules (which is about the second thing we do), we'll switch to a different container anyway.
The biggest question with that is whether we need to support site-specific service providers in the early installer, I don't remember ever using that feature but maybe someone does - have asked in slack.
This would probably be quite a lot of work to get right, but it would benefit every Drupal install (verb, not noun) in the wild, and regardless of whether installing from a profile, recipe, existing config etc. alongside all contrib functional and functional js tests, not just core pipelines (which is also the case for some of the other issues linked from here but not all).
Comment #6
catchComment #7
catchComment #8
nicxvan commentedShould this be expanded so we can add:
#3613347: Re-use container in kernel tests and #1411074: Add a flag to set up test environment only once per test class for kernel tests
Comment #9
catchYeah given it's just a tracking issue I don't think it hurts to expand the scope. This is already slightly expanding beyond container rebuilds with the .info.yml parsing and etc.