Create a set of migration files to enable users to migrate Drupal 7 Menu & Menu Links - ignoring system menus and system menu links

CommentFileSizeAuthor
#31 2382985-31.patch22.38 KBphenaproxima
#28 2382985-28.patch25.87 KBphenaproxima
#28 interdiff-2382985-26-28.txt1.91 KBphenaproxima
#26 2382985-26.patch24.42 KBphenaproxima
#25 2382985-24.patch35.22 KBquietone
#24 interdiff-2382985-21-23.txt2.59 KBquietone
#24 2382985-23.patch36.07 KBquietone
#21 interdiff-2382985-16-21.txt781 bytesquietone
#21 2382985-21.patch37.08 KBquietone
#16 interdiff-2382985-14-16.txt497 bytesquietone
#16 2382985-16.patch38.05 KBquietone
#14 interdiff-2382985-11-14.txt67.59 KBquietone
#14 2382985-14.patch38.04 KBquietone
#11 2382985-11.patch35.56 KBphenaproxima
#9 2392985-9.patch33.1 KBphenaproxima
#8 2382985-8.patch33.29 KBphenaproxima
#1 migration_files_for-2382985-1.patch1.8 KB-enzo-
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

-enzo-’s picture

-enzo- CreditAttribution: -enzo- commented
FileSize
migration_files_for-2382985-1.patch1.8 KB

I did a migration files to enable users to migrate menus and menu links from Drupal 7 to Drupal 8.

Is probable you are a Migration Exception error migration in you custom menu links Array index missing, extraction failed. sadly this message is not to mucho helpful about where is the problem.

After debug I determine the problem is located in process function of field options in D7 -> description in D8 the login require a strict multidimensional array in options field like this

[0] => array(
  ['attributes'] => array(
    'tittle' => ''
   )
)

To resolve this issue I did a script to be executed via Drush 

<?php

$query = db_select('menu_links', 'ml');
$query->fields('ml', array('mlid', 'options'));
$query->condition('module', 'menu');
$query->condition('ml.customized', 1);

$menu_links = $query->execute();

$default_options = array('attributes' => array('title' => ''));

foreach ($menu_links as $menu_link) {
  $options = unserialize($menu_link->options);
  // Include default options if they are missing
  $new_options = array_merge_recursive($default_options, $options) ;

  // Update menu link optios
  db_update('menu_links')->fields(
    array(
      'options' => serialize($new_options),
    ))->condition('mlid', $menu_link->mlid)->execute();

  print "Updated Menu link: " . $menu_link->mlid . "\n";
}

The script will create the required entries attributes or title inside attributes if doesn't exist. so your Drupal 7 will continue working and will be able to be migrated to Drupal 8

benjy’s picture

benjy CreditAttribution: benjy at CodeDrop commented
benjy’s picture

benjy CreditAttribution: benjy at CodeDrop commented
Parent issue: » #2456259: [META] Drupal 7 to Drupal 8 Migration path
Greg Boggs’s picture

Greg Boggs
he/him
Primary language English
Location Portland Oregon
CreditAttribution: Greg Boggs commented

Anyone know how to disable this migration when using the migration_UI to prevent it from causing fatal error the migration?

benjy’s picture

benjy CreditAttribution: benjy at CodeDrop commented

These migrations aren't in core, did you apply this patch manually? If so, just remove it.

I'm pretty sure the UI try's to detect the version you're running and then runs all migrations with that label, you could just remove this migration from your active store, or change the label should do it.

Greg Boggs’s picture

Greg Boggs
he/him
Primary language English
Location Portland Oregon
CreditAttribution: Greg Boggs commented

I get the same error in d6 link migration. How do I remove a migration from the active store? It's possible, I'm on the wrong issue even though the error message is the same.

To clarify, I did not apply this patch. This happens every time I run migrate UI for d6.

benjy’s picture

benjy CreditAttribution: benjy at CodeDrop commented

Sounds like you have a more general problem, might be best to create an issue on http://drupal.stackexchange.com/

phenaproxima’s picture

phenaproxima
he/him
Primary language English
Location Massachusetts
CreditAttribution: phenaproxima at Acquia commented
Status: Active » Postponed
FileSize
2382985-8.patch33.29 KB

Rerolled for 8.0.x HEAD and added tests. Since the menu_custom and menu_links tables are identical between D6 and D7, I also moved those source plugins out of version-specific namespaces.

That said, this is blocked by #2495755: Create MigrateDrupal7TestBase.

phenaproxima’s picture

phenaproxima
he/him
Primary language English
Location Massachusetts
CreditAttribution: phenaproxima at Acquia commented
Status: Postponed » Needs review
FileSize
2392985-9.patch33.1 KB

Status: Needs review » Needs work

The last submitted patch, 9: 2392985-9.patch, failed testing.

phenaproxima’s picture

phenaproxima
he/him
Primary language English
Location Massachusetts
CreditAttribution: phenaproxima at Acquia commented
Status: Needs work » Needs review
FileSize
2382985-11.patch35.56 KB

quietone queued 11: 2382985-11.patch for re-testing.

Status: Needs review » Needs work

The last submitted patch, 11: 2382985-11.patch, failed testing.

quietone’s picture

quietone CreditAttribution: quietone commented
Status: Needs work » Needs review
FileSize
2382985-14.patch38.04 KB
interdiff-2382985-11-14.txt67.59 KB

Mostly rerolled. MigrateMenuLinkTest for 7 will fail, complaining that skip_process_on_empty plugin doesn't exist. Let's see if testbot finds any other errors.

Status: Needs review » Needs work

The last submitted patch, 14: 2382985-14.patch, failed testing.

quietone’s picture

quietone CreditAttribution: quietone commented
FileSize
2382985-16.patch38.05 KB
interdiff-2382985-14-16.txt497 bytes

Add 'method: process' to d7_menu_links.yml

Not yet sure why d6\MigrateMenuLinkTest fails.

quietone’s picture

quietone CreditAttribution: quietone commented
Status: Needs work » Needs review

The last submitted patch, 8: 2382985-8.patch, failed testing.

The last submitted patch, 1: migration_files_for-2382985-1.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 16: 2382985-16.patch, failed testing.

quietone’s picture

quietone CreditAttribution: quietone commented
Status: Needs work » Needs review
FileSize
2382985-21.patch37.08 KB
interdiff-2382985-16-21.txt781 bytes

It was just one assertion, and the order of the array was changed. It is now reverted to the original.

quietone’s picture

quietone CreditAttribution: quietone commented

Two of the files in this patch contain 2 classes each, which contradicts PSR-4 namespaces and autoloading in Drupal 8 which states "Each PHP class, interface, or trait lives in a separate PHP file".

The two file are system/tests/src/Unit/Plugin/migrate/source/MenuTest.php and menu_link_content/tests/src/Unit/Plugin/migrate/source/MenuLinkSourceTest.php.

Have I got that right and this needs to be changed?

phenaproxima’s picture

phenaproxima
he/him
Primary language English
Location Massachusetts
CreditAttribution: phenaproxima at Acquia commented

Tests seem to be the exception to that rule. It's OK for test files to have more than one class, as long as that second class is a test-only double of the class being tested.

EDIT: After looking at the patch, you can actually remove both of those extra test classes. They are relics from a previous generation of testing and are no longer needed :)

quietone’s picture

quietone CreditAttribution: quietone commented
FileSize
2382985-23.patch36.07 KB
interdiff-2382985-21-23.txt2.59 KB

Yea, I should have checked if those test classes were needed.

quietone’s picture

quietone CreditAttribution: quietone commented
FileSize
2382985-24.patch35.22 KB

Remove 'use statements' that NetBeans shows are unused. Passes test locally.

phenaproxima’s picture

phenaproxima
he/him
Primary language English
Location Massachusetts
CreditAttribution: phenaproxima at Acquia commented
Category: Feature request » Task
Priority: Normal » Major
Issue tags: +Migrate critical
FileSize
2382985-26.patch24.42 KB

Merged the d6_menu and d7_menu migrations and did a bit of refactoring, removing some extraneous code.

Also escalating this to Migrate critical because menu links are going to be missed if they don't show up for people, and other migrations (like #2500513: Upgrade path for Shortcut 7.x) may depend on them.

Status: Needs review » Needs work

The last submitted patch, 26: 2382985-26.patch, failed testing.

phenaproxima’s picture

phenaproxima
he/him
Primary language English
Location Massachusetts
CreditAttribution: phenaproxima at Acquia commented
Status: Needs work » Needs review
FileSize
interdiff-2382985-26-28.txt1.91 KB
2382985-28.patch25.87 KB

Fixing the test failures.

benjy’s picture

benjy CreditAttribution: benjy commented 
  1. +++ b/core/modules/menu_link_content/migration_templates/d7_menu_links.yml
@@ -1,9 +1,9 @@
-id: d6_menu_links
-label: Drupal 6 menu links
+id: d7_menu_links
+label: Drupal 7 menu links

    Cool, git shows the diff between the file you copied from 

  2. +++ b/core/modules/menu_link_content/tests/src/Unit/Plugin/migrate/source/MenuLinkSourceTest.php
--- a/core/modules/migrate_drupal/config/schema/migrate_drupal.source.schema.yml
+++ b/core/modules/migrate_drupal/config/schema/migrate_drupal.source.schema.yml

    Are all the changes in this file intended?

benjy’s picture

benjy CreditAttribution: benjy commented 
+++ b/core/modules/migrate_drupal/config/schema/migrate_drupal.source.schema.yml
@@ -50,6 +50,154 @@ migrate.source.variable_multirow:
+migrate.source.menu_link:
+  type: migrate_source_sql

And this one should be in the migrate schema file, not the migrate_drupal one?

EDIT: Actually, not sure we need it at all anymore.

phenaproxima’s picture

phenaproxima
he/him
Primary language English
Location Massachusetts
CreditAttribution: phenaproxima at Acquia commented
FileSize
2382985-31.patch22.38 KB

#29.2: No, they weren't. Fixed in this patch.

ultimike’s picture

ultimike
Primary language English
Location Florida, USA
CreditAttribution: ultimike commented
Status: Needs review » Reviewed & tested by the community

I went through this with phenapromixa over my shoulder - assuming the testbot agrees, this looks good to go.

-mike

phenaproxima’s picture

phenaproxima
he/him
Primary language English
Location Massachusetts
CreditAttribution: phenaproxima at Acquia commented
Issue tags: -Migrate critical +Migrate D7 critical
webchick’s picture

webchick
she/they
Primary language English
Location Vancouver 🇨🇦
CreditAttribution: webchick at Acquia commented
Status: Reviewed & tested by the community » Fixed

The goofy stuff Git's doing here makes this diff a lot harder to understand than it needs to be. Basically:

phenaproxima
12:11 webchick: It's saying this: menus are identical for D6 and D7, so use the same thing for both. Menu *links* are 90% similar, but they still need separate migrations
However, their source plugins are identical

Code reuse FTW!

Committed and pushed to 8.0.x. Thanks!

  • webchick committed 142d048 on 
    Issue #2382985 by phenaproxima, quietone, -enzo-, benjy: Migration Files...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.