Problem/Motivation

Executing the Drupal 7 menu link migration (d7_menu_link) triggers MigrateExceptions thrown by the d7_menu migration:

Config entities can not be stubbed.

I dived into and tried to determine why Drupal throws these exceptions. After setting up a breakpoint in EntityConfigBase it turned out that every time this exception was thrown the migrate_lookup plugin tried to get the destination ID of a menu shortcut-set-<#>.

I searched for that string in core and it turned out that the menu links which have this kind of menu_names are handled exclusively by the d7_shortcut migration: The d7_shortcut migration source plugin explicitly filters for this string: because these menu links are migrated into shortcut entities.

The menu_link migration is logging several error messages concerning shourtcuts. See #12

Proposed resolution

Since there is an another migration for shortcut menu links, let's add an another condition to the menu_link migration source plugin's query that explicitly excludes shortcut links.

With this change the menu_link migration test will not be generating errors.

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

CommentFileSizeAuthor
#41 interdiff.3152943.38-41.txt604 bytesrocketeerbkw
#41 3152943-41.patch3.47 KBrocketeerbkw
#38 interdiff.3152943.39-38.txt5.37 KBrocketeerbkw
#38 3152943-38.patch3.47 KBrocketeerbkw
#38 3152943-38-fail.patch2.51 KBrocketeerbkw
#29 interdiff.3152943.26-29.txt1.5 KBabhisekmazumdar
#29 3152943-29.patch3.46 KBabhisekmazumdar
#26 interdiff_21-26.txt2.17 KBnikitagupta
#26 3152943-26.patch3.76 KBnikitagupta
#22 3152943-21.patch2.04 KBbbombachini
#22 3152943-21-fail.patch1.12 KBbbombachini
#21 3152943-20.patch2.04 KBbbombachini
#21 3152943-20-fail.patch1.12 KBbbombachini
#19 3152943-19.patch1.8 KBbbombachini
#19 3152934-19-fail.patch1 KBbbombachini
#13 3152934-13.patch2.02 KBayushmishra206
#13 interdiff_10-13.txt1.99 KBayushmishra206
#10 interdiff_5-10.txt944 byteschandrashekhar_srijan
#10 3152943-10.patch2.02 KBchandrashekhar_srijan
#2 core-menu_link_migration_cannot_stub_config_entities-3152943-2.patch790 byteshuzooka
#5 core-menu_link_migration_cannot_stub_config_entities-3152943-5.patch939 byteshuzooka
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

huzooka created an issue. See original summary.

huzooka’s picture

Assigned: huzooka » Unassigned
Issue summary: View changes
Status: Active » Needs review
FileSize
790 bytes
Wim Leers’s picture

+++ b/core/modules/menu_link_content/src/Plugin/migrate/source/MenuLink.php
@@ -21,7 +21,8 @@ class MenuLink extends DrupalSqlBase {
+      ->condition('ml.menu_name', 'shortcut-set-%', 'NOT LIKE');

🤓 Let's add an @see to point to the place where these are then handled.

huzooka’s picture

Assigned: Unassigned » huzooka
Status: Needs review » Needs work

👍

huzooka’s picture

Assigned: huzooka » Unassigned
Status: Needs work » Needs review
FileSize
939 bytes
mikelutz’s picture

Status: Needs review » Needs work
Issue tags: +Needs tests

This definitely needs some sort of tests. There are three shortcut-set- links in the core d7 test fixture, and they don't cause any issues with testbot, so there must be something more happening here that needs to be investigated and documented.

huzooka’s picture

Well, those links cause exceptions, but the exceptions are catched and are logged instead.

Wim Leers’s picture

Status: Needs work » Needs review
mikelutz’s picture

Status: Needs review » Needs work

We still need some sort of test to show what we are fixing.

chandrashekhar_srijan’s picture

Status: Needs work » Needs review
FileSize
2.02 KB
944 bytes
Wim Leers’s picture

@chandrashekhar_srijan: Could you also upload a test-only patch to verify that that is failing? 😊🙏

quietone’s picture

Status: Needs review » Needs work
  1. +++ b/core/modules/menu_link_content/src/Plugin/migrate/source/MenuLink.php
    @@ -21,7 +21,10 @@ class MenuLink extends DrupalSqlBase {
    +      // Shortcut set links are migrated by the "d7_shortcut" migration.
    

    The MenuLink source plugin is used for D6 and D7 sources. Can we expand this comment to state how this applies to D6?
    And a small nit, I am not sure we need to quote d7_shortcut.

  2. +++ b/core/modules/menu_link_content/tests/src/Kernel/Migrate/d7/MigrateMenuLinkTest.php
    @@ -141,6 +141,12 @@ public function testMenuLinks() {
    +    // Test shortcut menu links exclusion.
    +    $menu_link = MenuLinkContent::load('244');
    +    $this->assertEqual($menu_link, NULL);
    +    $menu_link = MenuLinkContent::load('473');
    +    $this->assertEqual($menu_link, NULL);
    

    The shortcut link will not exist with or without the patch so this isn't going to test for the attempt to stub a config entity. Instead of this we can test for the presence of the migrate messages, 'Config entities can not be logged' from the d7_menu migration by checking the message table. Turns out there are 4 of those without the patch and 0 with the patch.

    I think this is a better way to test that,

        // Test there have been no attempts to stub a shortcut in a MigrationLookup
        // process.
        $messages = $this->getMigration('d7_menu')->getIdMap()->getMessages()->fetchAll();
        $this->assertCount(0, $messages);
    
ayushmishra206’s picture

Status: Needs work » Needs review
FileSize
1.99 KB
2.02 KB

Addressed 14.1 and 14.2. @quietone can you please elaborate on this "Can we expand this comment to state how this applies to D6?"

quietone’s picture

@ayushmishra206, the question is will this break Drupal 6 migrations? I don't know if Drupal 6 has the same functionality for shortcuts as Drupal 7. Does it? The Drupal 6 fixture does not have any shortcuts but that is not proof to me. Looking at the query itself, it shouldn't be a problem. I just think it is prudent to ask the question and explain the answer, whatever it is, in the comment.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

quietone’s picture

Issue summary: View changes
Status: Needs review » Needs work
Issue tags: -Needs tests

Ah, so there are no shortcuts in D6. So, while the comment could state that but it is also not necessary.

What needs to happen here is to upload a fail patch (a patch with just the test) to provide proof that the fix is actually working. The working patch will also need to be uploaded again or this will sit at NW. Remember to upload the fail patch first and then the working patch.

Thanks.

quietone’s picture

Issue tags: +Novice

Adding tag novice for the work outlined in #16.

When making a fail patch, the patch name will be like 3152934-13-fail.patch, with the correct comment number of course. Then load the fail patch first followed by the complete/good patch. That way the good patch is run last and the issue isn't automatically set to NW but the testbot.

quietone’s picture

bbombachini’s picture

I hope I have understood the task, sorry if not.

bbombachini’s picture

bbombachini’s picture

Ugh, my bad, uploaded a patch relative to the module and not core, sorry, fixing that now.

bbombachini’s picture

bbombachini’s picture

bbombachini’s picture

Status: Needs work » Needs review
quietone’s picture

Status: Needs review » Needs work

Looking good.

  1. +++ b/core/modules/menu_link_content/src/Plugin/migrate/source/MenuLink.php
    @@ -21,7 +21,10 @@ class MenuLink extends DrupalSqlBase {
    +      // Shortcut set links are migrated by the d7_shortcut migration.
    

    Since this source plugin is used by D6 source as well we should comment that d6 doesn't have short cuts so this change will not effect d6 sources.

  2. +++ b/core/modules/menu_link_content/src/Plugin/migrate/source/MenuLink.php
    @@ -21,7 +21,10 @@ class MenuLink extends DrupalSqlBase {
    +      ->condition('ml.menu_name', 'shortcut-set-%', 'NOT LIKE');
    

    Ah, since this changes the source plugin query the source plugin test should change as well, \Drupal\Tests\menu_link_content\Kernel\Plugin\migrate\source\MenuLinkTest. I think it is sufficient to add a shortcut to the menu_link table to the existing test[0]. But then a fail patch will be needed for proof.

nikitagupta’s picture

Status: Needs work » Needs review
FileSize
3.76 KB
2.17 KB

Addressed 25.1,2.

Status: Needs review » Needs work

The last submitted patch, 26: 3152943-26.patch, failed testing. View results

quietone’s picture

  1. +++ b/core/modules/menu_link_content/src/Plugin/migrate/source/MenuLink.php
    @@ -21,7 +21,11 @@ class MenuLink extends DrupalSqlBase {
    +      // Shortcut set links are migrated by the d7_shortcut migration.
    +      // d6 doesn't have short cuts so this change will not effect d6 sources.
    

    I think something like this would be better

    Shortcut set links are migrated by the d7_shortcut migrations. Shortcuts are not used in Drupal 6.

  2. +++ b/core/modules/menu_link_content/tests/src/Kernel/Plugin/migrate/source/MenuLinkTest.php
    @@ -17,7 +17,7 @@ class MenuLinkTest extends MigrateSqlSourceTestBase {
    +  protected static $modules = ['menu_link_content', 'migrate_drupal', 'shortcut'];
    

    The shortcut module does not need to be installed. This IS explains that here we are removing getting short cuts from the source database.

  3. +++ b/core/modules/menu_link_content/tests/src/Kernel/Plugin/migrate/source/MenuLinkTest.php
    @@ -26,7 +26,7 @@ public function providerSource() {
    +    $tests[0]['source_data']['menu_links']['shortcut-set-2'] = [
    
    @@ -245,10 +245,10 @@ public function providerSource() {
    +    $tests[0]['source_data']['menu_links'][0]['options']['attributes']['title']['shortcut-set-2'] = $title;
    ...
    +    $expected = $tests[0]['source_data']['menu_links']['shortcut-set-2'];
    

    Instead of this (these changes need to be reverted) what is needed here is to prove that the query does not return menu rows that are short cuts. A new row needs to be added to 'menu_links' to do that. If we look at the query itself we see that rows with a particular name are ignored. Therefor the new row must have a name like that.

abhisekmazumdar’s picture

Status: Needs work » Needs review
FileSize
3.46 KB
1.5 KB

Hey @quietone
I have made the change as per the comment on #28 but I'm not sure about the changes for 28.3.
I understood that we need to revert the changes done for $tests[0]. And add a new row to menu_links. Also found out that the 5th row is been ignored. But I'm not sure what's needed to be in the newly added row.

Status: Needs review » Needs work

The last submitted patch, 29: 3152943-29.patch, failed testing. View results

quietone’s picture

@abhisekmazumdar, good question. I didn't explain that too well so here is more detail.

The array $tests[0]['source_data']['menu_links'] contains 7 arrays which represent a row in the 'menu_links' table that is used during the test. That is the data that the query changed here will use. Currently, none of those rows represents a short cut set so we don't have proof that the query will do what we want, which is to exclude menus that are short cut sets.

We actually have two options a) to add a whole set of test data in $tests[1] or b) add a row that represents a short cut set to the existing tests, $tests[0]. I think option b is sufficient which means that an 8th array needs to be added to $tests[0]['source_data']['menu_links']. We don't need to change the expected data, or any of the other input data to the test. We just need to make sure the 8th array represents a short cut row. If we look at the query itself it is excluding rows with where menu_name is 'shortcut-set-%'. That means that a menu_link for a short cut has a menu_name something like 'shortcut-set-foo', the value of the rest of the array keys doesn't matter. So, it is pretty much a copy/paste job to add a new array and change the menu_name.

Now, you mentioned the 5th row. That is excluded from the results because the router path is customized. It is this condition.
->condition('ml.router_path', ['admin/build/menu-customize/%', 'admin/structure/menu/manage/%'], 'NOT IN');

quietone’s picture

Title: Menu link migration tries to migrate shortcut links, and it ends in a MigrateException » Remove migration of shortcuts from menu_link migration

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

larowlan’s picture

Issue summary: View changes
Issue tags: +Bug Smash Initiative

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

rocketeerbkw’s picture

Issue summary: View changes
Status: Needs work » Needs review
FileSize
2.51 KB
3.47 KB
5.37 KB

I changed the test method per #31.

The last submitted patch, 38: 3152943-38-fail.patch, failed testing. View results

quietone’s picture

Status: Needs review » Needs work

@rocketeerbkw, thanks for updated patch.

One last small thing.

+++ b/core/modules/menu_link_content/tests/src/Kernel/Plugin/migrate/source/MenuLinkTest.php
@@ -241,6 +241,37 @@ public function providerSource() {
+        // D7 shortcut set link

Comments needs a period at the end.

rocketeerbkw’s picture

Status: Needs work » Needs review
FileSize
3.47 KB
604 bytes

Thanks! I didn't bother uploading another fail-only patch, see #38.

mikelutz’s picture

Status: Needs review » Reviewed & tested by the community

Took a look at this, it makes sense to me that we should be excluding these items here. Patch LGTM

quietone’s picture

Assigned: Unassigned » quietone

To other committers, I'll do this one in a few days.

quietone’s picture

Title: Remove migration of shortcuts from menu_link migration » Remove migration of shortcuts from menu_link migration

Updating title and credit

  • quietone committed 679dff2a on 10.1.x
    Issue #3152943 by bbombachini, rocketeerbkw, huzooka,...
quietone’s picture

This is changing the behavior of a source plugin such that shortcut links are not returned. In this case, that is not an problem because the saving a shortcut with the menu_link destination causes an error. This is cleaning up that error. I originally suggested that the assertion test for the text of the error message. Instead, the patch checks that there are zero error messages. That may be slightly better in that it increases the likelihood of failing due to other introduced errors.

Committed 679dff2 and pushed to 10.1.x. Thanks!

  • quietone committed 833765ae on 10.0.x
    Issue #3152943 by bbombachini, rocketeerbkw, huzooka,...

  • quietone committed 0e782309 on 9.5.x
    Issue #3152943 by bbombachini, rocketeerbkw, huzooka,...
quietone’s picture

Assigned: quietone » Unassigned
Status: Reviewed & tested by the community » Fixed

And cherry picked to 10.0.x and 9.5.x

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.