Problem/Motivation
As noticed in #2289201: [Meta] Make drupal install and run within reasonable php memory limits so we can reset the memory requirements to lower levels, using the fast chained cache backend during installation, when there are a lot of invalidations and cache writes results in a huge amount of merge queries due to marking the cache as invalid.
Proposed resolution
The easiest fix would be to add a check for the maintenance mode constant in the factory.
Another thought would be to do it in the factory, and possibly even default to the memory backend, as we invalidate most caches anyway with every installed module?
Remaining tasks
User interface changes
API changes
Comment | File | Size | Author |
---|---|---|---|
#11 | skip-fast-chained-during-install-2422657-11.patch | 835 bytes | Berdir |
#3 | skip-fast-chained-in-maintenance-mode-2422657-3.patch | 877 bytes | Berdir |
#1 | skip-fast-chained-in-maintenance-mode-2422657-1.patch | 4.18 KB | Berdir |
Comments
Comment #1
BerdirHere's a patch.
$ time drush si -y standard
HEAD:
Patch:
Not a huge difference, but I imagine the difference gets bigger with non-optimized/slower systems and bigger installation profiles.
Comment #2
amateescu CreditAttribution: amateescu commentedThe patch would be perfect if it wouldn't include #1768526: NodeFormController::validate() calls buildEntity() twice :P
Comment #3
BerdirPerfect is overrated anyway ;)
Comment #4
amateescu CreditAttribution: amateescu commentedIf we target only the installation process here, shouldn't we use
drupal_installation_attempted()
instead?Comment #5
catchJust thinking through.
-> site goes into maintenance mode, fast backend gets ignored, this is fine.
-> site goes out of maintenance mode, does that need to explicitly invalidate the fast backend at that moment?
Hadn't seen #4 when writing this, but similar question.
Comment #6
BerdirInteresting, I'm not sure how to solve that. Wouldn't the same happen if you e.g. do a drush cache clear and you don't have apcu enabled there? I've never seen any issues so far with that, but I might have apcu enabled.
Comment #7
catchHmm that's interesting and seems like a possible bug yes.
Comment #8
BerdirAny idea on how to prevent that? I can't think of something right now, the only thing I can think of right now is to do an immmediate invalidation if we get to that point once per request. This might work here, but I'm not sure about drush.
Also, if we limit this to just the installer, do we still need to worry about this or can we assume that the fast cache is empty at this point? I guess it could be a problem if you do multiple repeated installs?
Comment #9
Fabianx CreditAttribution: Fabianx commentedIn general this looks okay and Drupal traditionally has many caches off during Maintenance mode.
But it feels very very wrong to reference a global constant encapsulated in a class ...
I am unsure if it possible to do this differently though.
Comment #10
BerdirWell, the installer flag is a constant, it's not in the scope of this issue to change that.
The only alternative is using the drupal_installation_attempted() function, which isn't better in that regard.
What if we limit this to just the installer and not all maintenance modes? Do we still have to worry about stale caches in the fast backend? I guess if you re-install multiple times then it could happen. We're not seeing that in the tests right now, but I suspect that is because every test has a different configuration path, and therefore a different sitePrefix.
Comment #11
BerdirSo this was not even used in the in-test installations, lets see if this breaks anything. this might also speed up web tests now?
$ time php core/scripts/run-tests.sh --url http://d8/ --class "Drupal\user\Tests\UserSaveTest"
This seems to be slightly faster, 7.05-7.10s with HEAD vs. ~6.875s with the patch. Note that enabling more modules won't change anything, as those aren't added during the installer.
Comment #12
amateescu CreditAttribution: amateescu commentedThe testbot seems happy with the last patch so I think this is good to go.
Comment #13
alexpottThis issue is a major task that will improve performance significantly and it introduces no disruption. Per https://www.drupal.org/core/beta-changes, this is a good change to complete during the Drupal 8 beta phase. Committed a53a61f and pushed to 8.0.x. Thanks!
Comment #15
Fabianx CreditAttribution: Fabianx commentedI created #2427823: ChainedFastBackendFactory depends on drupal_installation_attempted() after speaking with catch to clean up the dependency of the OO code to non-OO code introduced here that makes e.g. unit testing impossible or difficult ...
Not sure what we should do, but I think a container variable should be good or something like that ...
Comment #16
Fabianx CreditAttribution: Fabianx commentedOpened #2427861: Fast chained backend is incompatible with cli that has apc.enable_cli = 0 (major) after a discussion with catch on this patch. Though this bug was present before.
With the original scope of this issue any MAINTENANCE MODE cache clears would have not propagated to when the site was put back active.