Problem/Motivation
While working on #2225681: Migrate D6 i18n blocks translated strings I made some changes to the Drupal6 database dump via the UI as required by that issue. One of the changes was unnecessary, just left over from me learning how to recreate the problem over there. Later, that change was causing failures and since it was not needed I removed it. This issue is to investigate the failure caused by that data.
The data was added in comment 176. It was all related to this entry in i18n_blocks and the relevant menu entries.
4 | menu | menu-fr-secondary-links | 0 | fr
MariaDB [dev2_dump]> select * from i18n_blocks;
+------+--------+-------------------------+------+----------+
| ibid | module | delta | type | language |
+------+--------+-------------------------+------+----------+
| 4 | menu | menu-fr-secondary-links | 0 | fr | migrated
+------+--------+-------------------------+------+----------+
4 rows in set (0.00 sec)
The error was reported in the full migration test, https://www.drupal.org/pift-ci-job/946662
1) Drupal\Tests\migrate_drupal_ui\Functional\d6\MigrateUpgrade6Test::testMigrateUpgradeExecute
Exception: Warning: array_walk() expects parameter 1 to be array, string given
Drupal\Core\Utility\LinkGenerator->generate()() (Line: 119)
It turns out that the failure has nothing to do with i18n_blociks. The failure is caused because in Drupal 6 query options are stored as a string and the migration does not convert it to an array. Then, when trying to load that menu in admin/structure/menu/manage/secondary-links a failure occurred.
A better title would be good too.
Tracked this done. It is caused because the in D6 the query options is a sting and it is an array in D8+ See #3.
Proposed resolution
Add a new process plugin link_options to convert the query string to an array. Use the processing in the \Drupal\migrate\Plugin\migrate\process\Route process plugin.
Remaining tasks
Patch
Test
Review
User interface changes
N/A
API changes
N/A
Data model changes
N/A
Comment | File | Size | Author |
---|---|---|---|
#27 | 2975461-27.patch | 6.66 KB | quietone |
#27 | interdiff-22-27.txt | 889 bytes | quietone |
#22 | 2975461-21.patch | 6.66 KB | quietone |
#22 | 2975461-21-fail.patch | 4.8 KB | quietone |
#21 | 2975461-21.patch | 6.66 KB | quietone |
Comments
Comment #2
quietone CreditAttribution: quietone as a volunteer commentedI have been able to reproduce the error with the test fixture by running a full migration, I did it from /upgrade, from a D8 site created from a standard install and all translation modules enabled. After the migration navigating to
/admin/structure/menu/manage/secondary-links?destination=/admin/structure/menu shows this warning
This is the same error I was getting when the data from the other patch was used, although it was reporting an error on the link added by that patch, that is it wasn't 'Test 2'. Still it is the same problem. The failure occurs on the two menu links that in the source db have a langcode in the options column. The only menu_links with a langcode are mlid 138 and 463, shown below.
Both menu links 138 and 463 have been migrated with a langcode of 'en'.
Also, there is another problem. Two menu links, the links to the two translated nodes, fail to migrate. The destid is NULL and 3 is the row status in the map table.
I am not familiar with the node translation migration. Is this intended or should those links have been skipped?
Comment #3
quietone CreditAttribution: quietone as a volunteer commentedThere are several things causing problems.
1) The array walk problem is caused because the query parameter is a string instead of an array in link/options. This may be a simple fix.
The migration simply copies the options array as is from Drupal 6 to link/options,
'link/options': options
, where the query parameter is a string. Fortunately, the route plugin converts the query options to an array, so it might work to just simply move the link/options migration to after the route process plugin and change it to'link/options': '@route/options'
.I tested with that and the only change to link/options was making the query parameters and array instead of string and there is no more warning message.
2) A French only menu link is migrated as 'en' because the langcode field in the options array is ignored. It will be simple to check that but it needs to work correctly for sites with and without language enabled.
3) The menu_links for the translated nodes are migrated as NULL and errors in the migrate_map. Can these be skipped instead so there is not error?
Comment #5
aiphesIs my issue could be related to menu links or pathologic ?
In the D8 db I get:
Le filtre pathologic:0 n'a pas pu être associé à un plugin de filtre existant ; repli sur filter_null.
in the
migrate_message_d6_filter_format
table.This is my link generator error, that break cron among other things like form field setting.
https://www.drupal.org/project/drupal/issues/3000935
Comment #8
matthandIn a Drupal 7 to Drupal 8 migration, any menu links that were inactive in Drupal 7 do not migrate. They give the same message found above that the path failed validation.
Comment #12
quietone CreditAttribution: quietone as a volunteer commentedHere is a fix for the original problem. The solution is to convert the query options from D6, which is a string, to an array.
There are two other problems that are identified in #3. They should be in a separate issue, or they are a duplicate of an existing one. I have not checked.
Comment #13
quietone CreditAttribution: quietone as a volunteer commentedTested with the D6 fixture and this works fine and the admin page the secondary menu link loads without error. Test passes locally too.
Comment #16
quietone CreditAttribution: quietone as a volunteer commentedOh, I didn't notice that it was the Translation test that failed.
This patch updates the Translation tests and includes some comment fixes.
Comment #17
quietone CreditAttribution: quietone as a volunteer commentedComment #18
quietone CreditAttribution: quietone as a volunteer commentedThis is at least a major
Comment #19
LendudeDiscussed the needed test coverage a bit with @quietone in #bugsmash, kernel tests are sufficient for these changes.
So with that in mind, this looks good to me.
Comment #20
MatroskeenThe patch looks good! Thanks @quietone!
I have read the issue summary and didn't catch the relation between
i18n_blocks
table and string to array conversion of menu link query parameter. If it's just a background, then we should probably highlight what is the actual issue. I'm adding some information below (hopefully it's helpful) that can be incorporated into the IS.It seems that link query support was added in 6.x version: #90570: Support fragment only links. At that time, it was just a string being saved into database.
Later on, it was changed to the array here: #700336: Cannot create or list menu items with Clean URLs off.. It was already 7.x.
That's why we have the following difference in the databases:
7.x (from fresh d7 copy):
a:2:{s:5:"query";a:1:{s:3:"foo";s:3:"bar";}s:10:"attributes";a:1:{s:5:"title";s:0:"";}}
6.x (from db fixture):
a:2:{s:5:"query";s:7:"foo=bar";s:10:"attributes";a:1:{s:5:"title";s:16:"Test menu link 2";}}
I have 2 questions:
1) Do we need to create any follow-up tasks for issues mentioned in #3?
2) Do we want to test link options after D7 migration? I noticed that we don't have links with "query" in D7 database fixture. In this case, we will prove it works fine for D7 as well.
Comment #21
quietone CreditAttribution: quietone as a volunteer commented#20.1 Regarding the points in #3, you are right I didn't comment on them (although I thought I had).
#20.2. I changed an existing test link in the D7 fixture to have query options and added that to the existing D7 test. It turns out that D7 stores the query options as an array instead of a string as in D6. So, no code changes are needed there. I have remade the fail patch to show that Drupal 7 tests for the query options pass without code changes.
Comment #22
quietone CreditAttribution: quietone as a volunteer commentedForgot to upload the fail patch. Starting over.
Comment #23
quietone CreditAttribution: quietone as a volunteer commentedThat is odd, I didn't change the status.
Comment #25
quietone CreditAttribution: quietone as a volunteer commentedThe fail patch contains a test for Drupal 7 and it passes, only the Drupal 6 tests fail, which is as expected.
So, the patch in #21 is the patch in #16 with the addition of test coverage for Drupal 7. And the issue summary was updated. I think that covers everything in #20.
Comment #26
MatroskeenThanks!
I noticed one more grammar nit in the code comment:
I think we can change "query string is stored as a string" to "If the query parameters are stored as a string" to avoid duplicated "string" word.
Once it's done, I'm happy with RTBC.
Comment #27
quietone CreditAttribution: quietone as a volunteer commentedSure. And here is the fix for the comment.
Comment #28
MatroskeenThanks!
Comment #30
catchCommitted/pushed to 9.4.x and cherry-picked to 9.3.x, thanks!
Comment #32
huzookaThis patch also changed what is migrated from Drupal 7.
Comment #33
quietone CreditAttribution: quietone as a volunteer commentedI disagree. This patch improves the testing for Drupal 7 migrations and changes the fixture and a test file it does not change any code that executes migrations for Drupal 7 source.
Comment #34
quietone CreditAttribution: quietone as a volunteer commentedRestoring title. It will match the commit message now.