Problem/Motivation

While working on #2225681: Migrate D6 i18n blocks translated strings I made some changes to the Drupal6 database dump via the UI as required by that issue. One of the changes was unnecessary, just left over from me learning how to recreate the problem over there. Later, that change was causing failures and since it was not needed I removed it. This issue is to investigate the failure caused by that data.

The data was added in comment 176. It was all related to this entry in i18n_blocks and the relevant menu entries.
4 | menu | menu-fr-secondary-links | 0 | fr

MariaDB [dev2_dump]> select * from i18n_blocks;
+------+--------+-------------------------+------+----------+
| ibid | module | delta                   | type | language |
+------+--------+-------------------------+------+----------+
|    4 | menu   | menu-fr-secondary-links |    0 | fr       | migrated
+------+--------+-------------------------+------+----------+
4 rows in set (0.00 sec)

The error was reported in the full migration test, https://www.drupal.org/pift-ci-job/946662

1) Drupal\Tests\migrate_drupal_ui\Functional\d6\MigrateUpgrade6Test::testMigrateUpgradeExecute
Exception: Warning: array_walk() expects parameter 1 to be array, string given
Drupal\Core\Utility\LinkGenerator->generate()() (Line: 119)

It turns out that the failure has nothing to do with i18n_blociks. The failure is caused because in Drupal 6 query options are stored as a string and the migration does not convert it to an array. Then, when trying to load that menu in admin/structure/menu/manage/secondary-links a failure occurred.


A better title would be good too.

Tracked this done. It is caused because the in D6 the query options is a sting and it is an array in D8+ See #3.

Proposed resolution

Add a new process plugin link_options to convert the query string to an array. Use the processing in the \Drupal\migrate\Plugin\migrate\process\Route process plugin.

Remaining tasks

Patch
Test

Review

User interface changes

N/A

API changes

N/A

Data model changes

N/A

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

quietone created an issue. See original summary.

quietone’s picture

I have been able to reproduce the error with the test fixture by running a full migration, I did it from /upgrade, from a D8 site created from a standard install and all translation modules enabled. After the migration navigating to
/admin/structure/menu/manage/secondary-links?destination=/admin/structure/menu shows this warning

Warning: array_walk() expects parameter 1 to be array, string given in Drupal\Core\Utility\LinkGenerator->generate() (line 119 of core/lib/Drupal/Core/Utility/LinkGenerator.php).

Drupal\Core\Utility\LinkGenerator->generate('Test 2', Object) (Line: 94)
Drupal\Core\Render\Element\Link::preRenderLink(Array)

This is the same error I was getting when the data from the other patch was used, although it was reporting an error on the link added by that patch, that is it wasn't 'Test 2'. Still it is the same problem. The failure occurs on the two menu links that in the source db have a langcode in the options column. The only menu_links with a langcode are mlid 138 and 463, shown below.

MariaDB [dev2_dump]> select mlid, plid, menu_name, link_title, options  from menu_links where plid=139 or mlid=139 order by plid;
MariaDB [dev2_dump]> select mlid, plid, menu_name, link_title, router_path, options  from menu_links where plid=139 or mlid=139 order by plid;
+------+------+-----------------+-------------+-------------+-----------------------------------------------------------------------------------------------------------------+
| mlid | plid | menu_name       | link_title  | router_path | options                                                                                                         |
+------+------+-----------------+-------------+-------------+-----------------------------------------------------------------------------------------------------------------+
|  139 |    0 | secondary-links | Test 2      | admin       | a:2:{s:5:"query";s:7:"foo=bar";s:10:"attributes";a:1:{s:5:"title";s:16:"Test menu link 2";}}                    |
|  138 |  139 | secondary-links | Test 1      | user/login  | a:2:{s:10:"attributes";a:1:{s:5:"title";s:16:"Test menu link 1";}s:8:"langcode";s:2:"en";}                      |
|  463 |  139 | secondary-links | fr - Test 1 | user/login  | a:3:{s:10:"attributes";a:1:{s:5:"title";s:21:"fr - Test menu link 1";}s:8:"langcode";s:2:"fr";s:5:"alter";b:1;} |
+------+------+-----------------+-------------+-------------+-----------------------------------------------------------------------------------------------------------------+

Both menu links 138 and 463 have been migrated with a langcode of 'en'.

Also, there is another problem. Two menu links, the links to the two translated nodes, fail to migrate. The destid is NULL and 3 is the row status in the map table.

The path "internal:/node/11" failed validation.
The path "internal:/node/13" failed validation.

I am not familiar with the node translation migration. Is this intended or should those links have been skipped?

quietone’s picture

Title: Investigate LinkGenerator error » menu_link migration errors

There are several things causing problems.

1) The array walk problem is caused because the query parameter is a string instead of an array in link/options. This may be a simple fix.
The migration simply copies the options array as is from Drupal 6 to link/options, 'link/options': options, where the query parameter is a string. Fortunately, the route plugin converts the query options to an array, so it might work to just simply move the link/options migration to after the route process plugin and change it to 'link/options': '@route/options'.
I tested with that and the only change to link/options was making the query parameters and array instead of string and there is no more warning message.

2) A French only menu link is migrated as 'en' because the langcode field in the options array is ignored. It will be simple to check that but it needs to work correctly for sites with and without language enabled.

3) The menu_links for the translated nodes are migrated as NULL and errors in the migrate_map. Can these be skipped instead so there is not error?

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

aiphes’s picture

Is my issue could be related to menu links or pathologic ?
In the D8 db I get:
Le filtre pathologic:0 n'a pas pu être associé à un plugin de filtre existant ; repli sur filter_null.
in the migrate_message_d6_filter_format table.
This is my link generator error, that break cron among other things like form field setting.
https://www.drupal.org/project/drupal/issues/3000935

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

matthand’s picture

In a Drupal 7 to Drupal 8 migration, any menu links that were inactive in Drupal 7 do not migrate. They give the same message found above that the path failed validation.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

quietone’s picture

Here is a fix for the original problem. The solution is to convert the query options from D6, which is a string, to an array.

There are two other problems that are identified in #3. They should be in a separate issue, or they are a duplicate of an existing one. I have not checked.

quietone’s picture

Title: menu_link migration errors » Convert query string to array for d6 menu_link migration
Issue summary: View changes

Tested with the D6 fixture and this works fine and the admin page the secondary menu link loads without error. Test passes locally too.

The last submitted patch, 12: 2975461-12-fail.patch, failed testing. View results

Status: Needs review » Needs work

The last submitted patch, 12: 2975461-12.patch, failed testing. View results

quietone’s picture

Status: Needs work » Needs review
FileSize
2.46 KB
4.8 KB

Oh, I didn't notice that it was the Translation test that failed.

This patch updates the Translation tests and includes some comment fixes.

quietone’s picture

Issue summary: View changes
quietone’s picture

Priority: Normal » Major

This is at least a major

Lendude’s picture

Status: Needs review » Reviewed & tested by the community

Discussed the needed test coverage a bit with @quietone in #bugsmash, kernel tests are sufficient for these changes.

So with that in mind, this looks good to me.

Matroskeen’s picture

The patch looks good! Thanks @quietone!

I have read the issue summary and didn't catch the relation between i18n_blocks table and string to array conversion of menu link query parameter. If it's just a background, then we should probably highlight what is the actual issue. I'm adding some information below (hopefully it's helpful) that can be incorporated into the IS.

It seems that link query support was added in 6.x version: #90570: Support fragment only links. At that time, it was just a string being saved into database.
Later on, it was changed to the array here: #700336: Cannot create or list menu items with Clean URLs off.. It was already 7.x.

That's why we have the following difference in the databases:
7.x (from fresh d7 copy):
a:2:{s:5:"query";a:1:{s:3:"foo";s:3:"bar";}s:10:"attributes";a:1:{s:5:"title";s:0:"";}}
6.x (from db fixture):
a:2:{s:5:"query";s:7:"foo=bar";s:10:"attributes";a:1:{s:5:"title";s:16:"Test menu link 2";}}

I have 2 questions:
1) Do we need to create any follow-up tasks for issues mentioned in #3?
2) Do we want to test link options after D7 migration? I noticed that we don't have links with "query" in D7 database fixture. In this case, we will prove it works fine for D7 as well.

quietone’s picture

Issue summary: View changes
Status: Reviewed & tested by the community » Needs review
FileSize
1.54 KB
6.66 KB

#20.1 Regarding the points in #3, you are right I didn't comment on them (although I thought I had).

#20.2. I changed an existing test link in the D7 fixture to have query options and added that to the existing D7 test. It turns out that D7 stores the query options as an array instead of a string as in D6. So, no code changes are needed there. I have remade the fail patch to show that Drupal 7 tests for the query options pass without code changes.

quietone’s picture

Status: Needs review » Reviewed & tested by the community
FileSize
4.8 KB
6.66 KB

Forgot to upload the fail patch. Starting over.

quietone’s picture

Status: Reviewed & tested by the community » Needs review

That is odd, I didn't change the status.

The last submitted patch, 22: 2975461-21-fail.patch, failed testing. View results

quietone’s picture

The fail patch contains a test for Drupal 7 and it passes, only the Drupal 6 tests fail, which is as expected.

So, the patch in #21 is the patch in #16 with the addition of test coverage for Drupal 7. And the issue summary was updated. I think that covers everything in #20.

Matroskeen’s picture

Status: Needs review » Needs work

Thanks!

I noticed one more grammar nit in the code comment:

If the query string is stored as a string (as in D6), convert it into an array.

I think we can change "query string is stored as a string" to "If the query parameters are stored as a string" to avoid duplicated "string" word.
Once it's done, I'm happy with RTBC.

quietone’s picture

Status: Needs work » Needs review
FileSize
889 bytes
6.66 KB

Sure. And here is the fix for the comment.

Matroskeen’s picture

Status: Needs review » Reviewed & tested by the community

Thanks!

  • catch committed 55a804c on 9.4.x
    Issue #2975461 by quietone, Matroskeen, Lendude: Convert query string to...
catch’s picture

Status: Reviewed & tested by the community » Fixed

Committed/pushed to 9.4.x and cherry-picked to 9.3.x, thanks!

  • catch committed d63941b on 9.3.x
    Issue #2975461 by quietone, Matroskeen, Lendude: Convert query string to...
huzooka’s picture

Title: Convert query string to array for d6 menu_link migration » Convert query string to array for d6 and d7 menu_link migrations
Issue tags: +migrate-d7-d8

This patch also changed what is migrated from Drupal 7.

quietone’s picture

I disagree. This patch improves the testing for Drupal 7 migrations and changes the fixture and a test file it does not change any code that executes migrations for Drupal 7 source.

quietone’s picture

Title: Convert query string to array for d6 and d7 menu_link migrations » Convert query string to array for d6 menu_link migration
Issue tags: -migrate-d7-d8

Restoring title. It will match the commit message now.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.