Problem/Motivation

The d7_url_alias migration runs at an arbitrary time. This has been fine. Until recently

Recently, two things happened that make this problematic:

  1. It's now possible to enable validation for content entity type destinations #2745797: Add option to content entity destinations for validation
  2. Path aliases have been converted to entities: #3009014: Convert path alias migrations to use entity:path_alias destination and … added validation logic

Combined, this means that if you enable validation for content entity destinations (and you should — see #3095456: [Plan] Discuss strategy for long-term use of entity validation in the migration system), the d7_url_alias migration runs at a time when usually no content entities have been migrated yet. Result: an enormous amount of validation errors for PathAlias entities.

Proposed resolution

Automatically enhance migrations with Path Alias entities as their destination (destination plugin entity:path_alias) to have optional dependencies on every migration that has a locatable content entity type as their destination (if it's not locatable, it can't have an URL, and there's no point in pushing Path Alias entity creation until later).

In Drupal core, this would affect the d6_url_alias and d7_url_alias migrations. But it would also automatically enhance contributed/custom migrations.

Remaining tasks

Review.

User interface changes

None. Other than: far fewer validation errors when validations are enabled.

API changes

None.

Data model changes

None.

Release notes snippet

N/A

CommentFileSizeAuthor
#2 3098985-2.patch2.48 KBwim leers

Comments

Wim Leers created an issue. See original summary.

wim leers’s picture

wim leers
Location Ghent 🇧🇪🇪🇺
commented
Status: Active » Needs review
StatusFileSize
new 3098985-2.patch2.48 KB

Status: Needs review » Needs work

The last submitted patch, 2: 3098985-2.patch, failed testing. View results

heddn’s picture

heddn
Primary language English
Location Nicaragua
commented

An interesting note, the path alias migration is traditionally the slowest of all migrations to execute.

wim leers’s picture

wim leers
Location Ghent 🇧🇪🇪🇺
commented

#4: ohhh, that is interesting! Slower than node + node revision migrations? 🤯

Did the conversion of path aliases to entities make things slower? I would be super surprised if it didn't to be honest.

heddn’s picture

heddn
Primary language English
Location Nicaragua
commented

Continued side note: I haven't check with the conversion to entities. But traditionally, if you are migrating a site one of the first content migrations is users. Then path aliases float pretty high in run order. And running them tends to take a _really_ long time. And it blocks all successive migrations. Pretty often I tend to cancel that migration in my testing, which leaves it in a semaphore state that shows it is running and drush let's me move on to more interesting migrations. After I can confirm all my nodes and their field data are in fact migrating correctly, then I do the right thing and migrate the entire site's content per a more normal flow.

wim leers’s picture

wim leers
Location Ghent 🇧🇪🇪🇺
commented

Then path aliases float pretty high in run order. And running them tends to take a _really_ long time. And it blocks all successive migrations.

Why do path aliases need to run so early? Why does it block all successive migrations? That's not what I've seen so far, so I'm wondering what I'm missing 🤔

heddn’s picture

heddn
Primary language English
Location Nicaragua
commented

Still following the rabbit trail a little... but they don't "need" to. Its just that traditionally they didn't have any validation or any dependencies. Do to lack of dependencies, they ran pretty early. They can move later if there is a _need_. Now they have an entity destination, perhaps that needs has surfaced?

wim leers’s picture

wim leers
Location Ghent 🇧🇪🇪🇺
commented

Now they have an entity destination, perhaps that needs has surfaced?

Exactly :)

quietone’s picture

quietone commented

This is a different approach to dependencies, creating them as optional and in a hook migration_plugins_alter. Is this just a one off here or should we simply add the dependencies to the d6 an d7 url alias migrations? If we keep this it should be documented as to why it is different and the ymls need a comment as well.

Also, unlike other migrations this one adds dependencies on both drupal versions to d6_url_alias.yml and d7_url_alias.yml. Yes, they are added as optional so will work just fine.

  1. +++ b/core/modules/path/path.module
@@ -156,3 +156,52 @@ function path_field_widget_form_alter(&$element, FormStateInterface $form_state,
+  // Construct optional migration dependencies.
+  $optional_migration_dependencies = array_combine(
+    $locatable_entity_type_destination_migration_ids,
+    $locatable_entity_type_destination_migration_ids
+  );

    The dependencies are an indexed array, this is not needed. 

  2. +++ b/core/modules/path/path.module
@@ -156,3 +156,52 @@ function path_field_widget_form_alter(&$element, FormStateInterface $form_state,
+    $migrations[$id]['migration_dependencies']['optional'] += $optional_migration_dependencies;

    s/optional_migration_dependencies/locatable_entity_type_destination_migration_ids/

wim leers’s picture

wim leers
Location Ghent 🇧🇪🇪🇺
commented

Is this just a one off here or should we simply add the dependencies to the d6 an d7 url alias migrations?

It's because core cannot know which contrib and custom entity types a site has installed. This migration must run after all entity migrations.

If we keep this it should be documented as to why it is different and the ymls need a comment as well.

Agreed :) Does the above explanation make address your concerns? Happy to add it to the patch!

quietone’s picture

quietone commented

@Wim Leers, yes that explanation is helpful, thanks. I think comments should be added to d6_url_alias.yml, d7_url_alias.yml and path.module.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.7 was released on June 3, 2020 and is the final full bugfix release for the Drupal 8.8.x series. Drupal 8.8.x will not receive any further development aside from security fixes. Sites should prepare to update to Drupal 8.9.0 or Drupal 9.0.0 for ongoing support.

Bug reports should be targeted against the 8.9.x-dev branch from now on, and new development or disruptive changes should be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.2.x-dev

Drupal 8 is end-of-life as of November 17, 2021. There will not be further changes made to Drupal 8. Bugfixes are now made to the 9.3.x and higher branches only. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.15 was released on June 1st, 2022 and is the final full bugfix release for the Drupal 9.3.x series. Drupal 9.3.x will not receive any further development aside from security fixes. Drupal 9 bug reports should be targeted for the 9.4.x-dev branch from now on, and new development or disruptive changes should be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.9 was released on December 7, 2022 and is the final full bugfix release for the Drupal 9.4.x series. Drupal 9.4.x will not receive any further development aside from security fixes. Drupal 9 bug reports should be targeted for the 9.5.x-dev branch from now on, and new development or disruptive changes should be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

quietone’s picture

quietone commented
Status: Needs work » Postponed

The Migrate Drupal Module was approved for removal in #3371229: [Policy] Migrate Drupal and Migrate Drupal UI after Drupal 7 EOL.

This is Postponed. The status is set according to two policies. The Remove a core extension and move it to a contributed project and the Extensions approved for removal policies.

The deprecation work is in #3522602: [meta] Tasks to remove Migrate Drupal module and the removal work in #3522602: [meta] Tasks to remove Migrate Drupal module.

Migrate Drupal will not be moved to a contributed project. It will be removed from core after the Drupal 12.x branch is open.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.