Problem/Motivation
If db updates exist in an update, we need to decide what to do. We could:
- roll back the update
- run db updates
- put site in maintenance mode
- do nothing and hope for the best (site will maybe break, maybe it will be lucky and it won't break)
Proposed resolution
8x: default to putting site in maintenance mode, running db updates, then taking site out of maintenance mode. There is no UI for changing this (yet). But the order and sequence of how to handle db updates is configurable via config in automatic_updates.settings.yml. Alternatively, override via settings.php $config['automatic_updates.settings']['database_update_handling'] = ['rollback']. It is a an array of plugin IDs. Yes, db handling is done via plugins and a plugin manager.
Available plugins bundled in the module:
- 'execute_updates'
- 'ignore_updates'
- 'maintenance_mode_activate'
- 'maintenance_mode_disactivate'
- 'rollback'
A specific site (or agency or hosting provider) might decide to provide additional db handling options beyond these. For example, run a db dump and offload it to S3 or send emails/notices to slack. Then run the update.
7x: default to rolling back the upgrade if a database update exists. Very few db updates are bundled in drupal core at this point in its life-cycle. Safer to just rollback.
Lastly, of note is that a tricky part of the implementation of this feature set is that class loading/caching was a big hurdle. The only way around it was to shell out to a drush-like series of commands to determine if 1) db updates exist to execute 2) execute those db updates 3) flush cache. Cache flushing is needed as part of the db update process + it is always run after an upgrade, even no db updates exist.
For 8x, these are symfony console commands. For 7x, these are homegrown php shell scripts. Fewer options are available in 7x due to the lack of external dependencies.
The commands mirror drush very closely:
- updatedb [updb] Apply any database updates required (as with running update.php). (8x only)
- updatedb-status [updbst|updatedb:status] List any pending database updates.
- cache:rebuild [cr, rebuild] Rebuild a Drupal site and clear all its caches.
Remaining tasks
User interface changes
API changes
Data model changes
Release notes snippet
| Comment | File | Size | Author |
|---|---|---|---|
| #30 | 3093993-30.patch | 57.28 KB | heddn |
| #30 | interdiff_28-30.txt | 1.29 KB | heddn |
| #30 | 3093993-30-7x.txt | 10.99 KB | heddn |
Comments
Comment #2
heddnComment #3
heddnComment #4
ressaI think running any available db updates should be default, after enabling the module, to complete the security update. In theory, the database update itself could be doing the plugging of a security hole, right? Essentially a
drush up --security-only -y.(^^ options are radio buttons)
In which scenarios would we want to roll back the update? Or do you mean roll back, if something fails?
Comment #5
heddnNo UI yet, but let's see how badly this makes things.
Comment #6
ressaThe patch applies fine, and both Cron-triggered and the experimental link still updates the code from 8.7.4 to 8.7.9. But I get the error "The website encountered an unexpected error. Please try again later.", for both afterwards, and the db updates are not applied, but have to be applied manually.
Comment #7
heddnJust another WIP patch. Ignore the noise for now. Got to fix #3093782: Use an external csig file, not packaged inside zip archive first before we can keep moving here.
Comment #8
heddnNot sure how helpful the interdiff will be. This does work now, at least for rollbacks with manual testing from 8.7.5 and upgrade to 8.8.0-beta1. That manual testing demonstrated that some of the rolling back wasn't actually working as expected.
Still no UI. But there are now 4 "options" for handling database updates. These options are all plugins.
Comment #9
heddnThere's something a little odd going on where it always seems to think there are db updates and tries to roll things back. But incremental progress.
Comment #10
heddnComment #11
heddnHere's a 7.x patch while we're working on the test failures.
Comment #12
heddnComment #13
heddnThere are some serious issues with actually running database updates. Besides the philosophical ones, there's also technical ones that we're doing the entire update in a single PHP thread. Which means that class cache will have the old version of the classes; prior to updating. And its a bit of a catch-22, because you can't even flush cache or rebuild the container mid-flight or the site will start to 500. More time should be spent thinking about db updates. For now, if there's a db update, it defaults to rolling back the update.
Comment #14
heddnTo be more clear, don't use the
execute_updatesdb update handler because it currently doesn't work. Examples of errors while updating from 8.7.10 => 8.8.0-rc1:Call to undefined method Drupal\Core\File\FileSystem::scanDirectory()";s:9:"%function";s:53:"Drupal\Core\Asset\CssCollectionOptimizer->deleteAll()";s:5:"%file";s:113:"core/lib/Drupal/Core/Asset/CssCollectionOptimizer.phpRuntimeException: SplFileInfo::getMTime(): stat failed for core/modules/system/tests/modules/path_test/path_test.info.yml in SplFileInfo->getMTime()Argument 1 passed to Drupal\Core\Config\Entity\ConfigEntityUpdater::update() must be of the type array, null given, called in core/modules/text/text.post_update.php on line 47Comment #15
heddnSo, here we disable the cache clear in the db update. That get's a little further, but then we don't actually finish clearing cache ever and the site still doesn't work. A simple cache clear brings the site back to a functioning state. But until then, the site 500s.
Comment #16
heddnComment #17
heddnComment #18
heddnComment #19
andypostmoreover apcu cache (used by autoloader by default) probably also need clean-up
Comment #20
heddnRe #19, wouldn't that get caught by normal cache clearing? Apcu seems to be registered as a cache backend service (cache.backend.apcu).
Comment #21
heddnHere's a console command cache clear. We'll do it differently in D7.
Comment #22
andypostComment #23
heddnIf this passes green, I think it is ready for reviews.
Comment #24
heddnClean-up to remove debugging.
Comment #25
heddnRemoving more statements.
Comment #26
heddnComment #27
heddnGood, green.
Comment #28
heddnMore minor clean-up.
Comment #29
heddnComment #30
heddnComment #32
heddnAdding credit for assistance w/ the final approach.
Comment #35
heddnComment #36
ressaGreat job heddn, andypost and greg.1.anderson! I just updated from 8.7.4 to 8.7.10 by enabling "Enable automatic updates of Drupal core via cron" and running Cron. Also, there weren't any updates available at update.php, and this in the log files:
... so the available updates were applied.
I don't see any settings though, which is fine. But perhaps the Issue summary could have a bit of text about the solution which was decided on? Also the text on the admin/config/automatic_updates page says "Database updates are not run after an update.".
The only possible error I found was the warning I get after the update has run:
Flushing the cache didn't make a difference, but running the readiness check takes care of it. So perhaps it's working as designed, since the user should check if files are ready manually themselves, after an automatic update has been executed?
Comment #37
heddnOpened #3099319: Readiness checks results after an attempted cron upgrade can be wrong and #3099321: Remove module settings form copy about running database updates, Follow-up to #3093993 as follow-ups.
Comment #38
heddnComment #39
chi commentedTo make it compatible with Symfony 4.4+ command execute methods should return exit code.
See #3099295: Always return exit code in Symfony console commands for details.
Comment #40
heddnComment #41
heddnAdded #3099325: Always return exit code in Symfony console commands per #39. Thanks for the report.
Comment #42
ressaThanks for updating the Issue Summary. You all have really made a lot of progress, and it's coming along nicely. I'll test the patches in the other issues.