Problem/Motivation
I discovered a very annoying bug in #3219078: Regression: multilingual taxonomy term migrations are failing if user tries to migrate from Drupal 7.81+ and Entity Translation 7.x-1.1 which highlights that core migration tests are testing the migration of a very outdated Drupal 7 instance. While it seems that most of the contributed modules don't got data schema or value updates, settings up an outdated codebase (which works only with unsupported PHP versions) is a constant challenge. ...and it also gives us a false sense of "since core tests are passing, we definitely can migrate most of the Drupal 7 config + data into Drupal9". See #3219078: Regression: multilingual taxonomy term migrations are failing if user tries to migrate from Drupal 7.81+ and Entity Translation 7.x-1.1 .
Steps to reproduce
For an example check #3219078: Regression: multilingual taxonomy term migrations are failing if user tries to migrate from Drupal 7.81+ and Entity Translation 7.x-1.1
Proposed resolution
Update the core database fixture. this means:
- Scan the system table for module versions (assuming that it contains the right info).
- Build a codebase with the outdated core + contribs. Make sure you use the right PHP version.
- Import the DB fixture.
- Update the codebase with the most recent core and contribs.
- Execute the database updates.
- Identify which manual fixture edits caused the errors you see, and try to fix them.
- Repeat #3, #4 and #5 until you think that the database integrity is fixed.
- Export the updated DB and add that to core.
- Preferred: document the versions of core + contrib modules – I would add a drush make file for example.
Imho #3213633: Improve DX of maintaining migration database fixtures: provide an option for creating per-table database fixtures in DbDumpcommand can help us a lot, because it allows you to easily skip adding unnecessary changes made in tables on a per-table-basis, and not only on a per-hunk basis.
Remaining tasks
- Triage this issue
- Agree on a solution.
- Do it.
User interface changes
Nothing.
API changes
Nothing.
Data model changes
Nothing.
| Comment | File | Size | Author |
|---|---|---|---|
| #22 | 3219539-22-10.0.x.patch | 1.14 MB | quietone |
| #22 | interdiff-17-22-10.x.txt | 788 bytes | quietone |
| #18 | drupal7.make_.txt | 4.36 KB | quietone |
| #17 | 3219539-17.patch | 1.14 MB | quietone |
| #14 | 3219539-15.patch | 1.14 MB | quietone |
Comments
Comment #2
huzookaComment #3
quietone commentedI installed the fixture to my existing (outdated) Drupal 7 test site, adding anonymous and cleared cache with drush. Then exported and made a patch.
Comment #4
quietone commentedUpdated to Drupal 7.82, updated all modules and ran db updates.
Comment #5
quietone commentedI need to amend that. I did not update all contrib modules, I only did the ones that are enabled.
Comment #8
quietone commentedI've started over on this. I have updated to Drupal 7.92 and also the latest versions of all the installed modules. Just like before this db mostly works in a drupal 7 site. One thing to fix, is that viewing any of the article nodes still results in an error.
EntityReferenceRecursiveRenderingException: Recursive rendering detected when rendering entity node(3). Aborting rendering. in entityreference_field_formatter_view() (line 1420 of /var/www/html/drupal/sites/all/modules/entityreference/entityreference.module).Before addressing those I am running the tests.
Comment #9
quietone commentedComment #11
quietone commentedLooks like I forgot to add the Breakpoints and Picture modules. This add those which should fix all the texts except for MigrateNodeCounterTest.
This also fixes the format of Reference2 at admin/structure/types/manage/article/display. I learned from experience that this was prevented using the fixture from the UI on a D7 site.
Comment #13
quietone commentedThe modules on the source site were not in the right state, dashboard needed to be disabled, and syslog and testing needed to be enabled. This will not fix everything :-(
Comment #14
quietone commentedThis time disable picture module and breakpoint module. Also made sure to truncate accesslog and to restore the original node counter table data.
Comment #17
quietone commentedLooks like I didn't restore the node counter table. Trying again.
Comment #18
quietone commentedAssuming tests pass then the fixture has been updated. I've added a make file.
And for completeness noting other problems with the fixture. There problems existed before the update.
1) /admin/modules - Enabling a module or modules that require the use of the confirm form will cause many modules to be uninstalled.
2) Navigating to /is/firefly-is or /is/deep-space-9-is results in the error shown below. This can be fixed by changing the format of 'Reference 2' from 'Rendered entity' to 'Entity id' at admin/structure/types/manage/article/display. That would require a change to d7/MigrateFieldFormatterSettingsTest.php which is work with little gain. If the node needs to be edited just change the format.
EntityReferenceRecursiveRenderingException: Recursive rendering detected when rendering entity node(4). Aborting rendering. in entityreference_field_formatter_view() (line 1420 of /var/www/html/drupal/sites/all/modules/entityreference/entityreference.module).Comment #19
heddnAssuming this comes back green on 10.x and 10.1, this is RTBC.
Comment #20
alexpottIt didn't come back green on 10.x
Comment #21
mradcliffeA slightly more helpful diff of the fail using
s/, /\r/greveals:- Administer fields was added in Drupal 7.50 - https://www.drupal.org/node/2483307
- Dashboard and Overlay removed from Drupal 8 - https://www.drupal.org/node/2116417
Comment #22
quietone commentedI was going to test on D10 this morning but I see it has already happened and the results have been diagnosed. Thanks @heddn and @mradcliffe.
And here is the patch for D10.
Comment #23
heddn+1 on RTBC from me, given the original scope of things here. Can we improve the fixture or make it less complicated. Assuredly we could. But let's do that in a follow-up.
Comment #24
alexpottCommitted and pushed 7610d3c1ef to 10.1.x and 7225a0b56e to 10.0.x. Thanks!
Committed abde57f and pushed to 9.5.x. Thanks!
We have better tests on D10 thanks to #2953111: Only migrate role permissions that exist on the destination