Come together with the global Drupal community in Rotterdam, 28 Sept – 1 Oct 2026. Sessions, contribution, connection, and Early Bird savings until 8 June.Problem/Motivation
See #3305003: Create standard steps for creating database dumps and #3213633: Improve DX of maintaining migration database fixtures: provide an option for creating per-table database fixtures in DbDumpcommand for some background.
Right now we ship database dumps compressed with core. Updating these is complicated and error prone - i.e. to remove a module from a dump you have to load it into the version of core that the database dump was created with, uninstall the module, then re-export the dump. But the re-exporting might need to be made with the 11.x version of the script, not the 9.4.x version of the script, to incorporate improvements to the dump script. It's equally complicated updating for example an 8.9.x dump to a 9.4.x dump when we remove database updates.
I don't have a fully developed idea for how this would work, but I think we can possibly replace this if we rely on gitlab a bit more. See also this discussion in slack:
https://drupal.slack.com/archives/C1BMUQ9U6/p1700048273974479
1. Have a gitlab pipeline that installs, via a build test, the version of Drupal we want to update from. In the build test, create the content that we want to update (save nodes, comments, taxonomy terms, users, create some extra fields of various types etc.).
2. Somewhere in the build test, export the populated database to a dump file.
3. Save the dump file either as an artifact of the file itself, or into a container.
4. In the actual update test, we then rely on the database dump file, load it and update from that.
This way steps #1-3 can happen once per week, once per month etc. instead of every single time that tests run - so even though it'll be a lot of processing going on creating the dump, it'll happen infrequently.
To modify the dump output, we can update the test to add extra/different content and/or update the test to install a different version of Drupal.
We can also produce multiple database exports - for example a 9.4.4 dump, but also a 9.4.4 dump upgraded to 9.5.11. Or a 9.5.11 dump from a fresh install. If the tests work across core versions, this could even be done via a data provider.
Comments
Comment #2
catchComment #3
alexpottThe fun will be making them locally too, but this is a great idea. +1
Comment #4
catchComment #5
bbralaIf we maken sure running locally works by having the job in the main yaml, making it locally is easy, just takes some cpu.
Or you just download a dump
Comment #6
catchYes definitely #5 is the challenge, but I think with options both to recreate the dump or download one, then it should be OK - the instructions for that will be easier to write than the ones for editing a dump now.
Comment #7
wim leersAs the last person to having manually fixed the fixtures at #3400368-10: Deprecate path.temporary in system.file configuration schema, following 🤓
Comment #8
nicxvan commentedI know this was originally created for different reasons, but this is related to a public security issue so I hope it's ok to tag this and mark that issue as the parent.
Comment #9
catchSlowly learning how cross-pipeline artifacts work in #3462763: Use artifacts to share the phpstan result and cspell caches from core to MRs and it's all a bit tricky but it should work for this.
Comment #10
hetal.solankiComment #11
catchDiscussed this briefly with @alexpott and DrupalCon Vienna and he had a similar but potentially much easier idea than what's currently in the issue summary.
Instead of using a build test to generate a database dump on an older Drupal version, we could use a build test to install Umami, and then update that installed site.
We can probably install modules and create content on the installed site prior to updating it if we need extra data to test a specific update too.