Problem/Motivation

In HEAD, the d7_node_title_label migration runs much later than all d7_node:* migrations. While not wrong, this is … confusing.

In HEAD, I first have to run d7_node_type, but then I can run d7_node:* before d7_node_title_label. Which means that if I go to /admin/structure/types/manage/client to inspect my data model prior to importing the actual data, I see this:

It also means that if I run my migration with entity validation enabled (#2745797: Add option to content entity destinations for validation), that the validation errors use the incorrect field label.

Proposed resolution

Refine the migration dependencies, so we see this instead:

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

Comments

Wim Leers created an issue. See original summary.

wim leers’s picture

Status: Active » Needs review
StatusFileSize
new496 bytes

Status: Needs review » Needs work

The last submitted patch, 2: 3096951-2.patch, failed testing. View results

wim leers’s picture

Title: d7_node migration should have optional dependency on d7_node_title_label migration » d7_node migration should have dependency on d7_node_title_label migration
Issue summary: View changes
Related issues: +#2745797: Add option to content entity destinations for validation
quietone’s picture

Status: Needs work » Needs review
Issue tags: +DrupalSouth 2019
StatusFileSize
new7.35 KB

Added 'd7_node_title_label' to list of migrations to be run in all the failing tests. No interdiff because the previous patch is just one line and an interdiff would be noise.
Let's see what fails.

Status: Needs review » Needs work

The last submitted patch, 5: 3096951-5.patch, failed testing. View results

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new8.77 KB
new2.04 KB

Fixing the tests.

wim leers’s picture

Looks good to me :) Somebody else should RTBC this though.

wim leers’s picture

heddn’s picture

Issue tags: +Needs tests

Feels like we want/need a test to make sure that only well known Configuration migrations run /after/ content migrations. Meaning, we have this config migration running very late in the stage for no good reason. Some config migrations need to. But not this one. Ideally, migrations that are tagged configuration would run much earlier. But we don't have any test that is doing any testing in this space. So it could easily be that there are other config migrations running later then we want as well. Based on the work in #3099079: Generate graphs of migration dependencies, perhaps we could beg/borrow some of that code and pull it into here (or another task this is pended on) to build out a test of these things.

mikelutz’s picture

I'm opposed to making adding this dependency, simply because there is not an actual dependency between the migrations. Regardless of which order you run the migrations, after running both your system is in the same state. By definition, there is no dependency. There are many pieces required to migrate the entire node system into Drupal 8 from Drupal 7, and they are all required to be complete for the full system to match the state of the old installation. Intuitively it FEELs like the d7_node content somehow needs to be last, but it just doesn't. d7_node does not depend on d7_node_title_label, a full migration simply depends on them both, in any order.

In the sense of the core migration suite, dependencies should be rare and used sparingly. Their main use is when we are migrating configurations that depend on other configurations. Adding additional unneeded dependencies makes it more difficult for developers to run custom migration suites. They should be able to pick and choose what they want to migrate, and not have to deconstruct a complex and arbitrary dependency graph to figure what dependencies are actually required, and which are there just because we feel a little better about the order they run.

In HEAD, I first have to run d7_node_type, but then I can run d7_node:* before d7_node_title_label. Which means that if I go to /admin/structure/types/manage/client to inspect my data model prior to importing the actual data, I see this:

This is not possible to do in core. There is no mechanism to pick and choose migrations to run, then stop to inspect a data model prior to importing the data. Core runs all the migrations after sorting needed dependencies, and at the end of that run, it doesn't matter where the d7_node_title_label migration runs, as long as it's after d7_node_type. This is accurately reflected in the dependencies.

If you are doing this using the contrib migration add-ons, and you want to inspect configurations prior to migrating content, then you should be running the migrations with drush mim --tags=Configuration, which will cause the d7_node_title_label migration to be ran. Beyond that, if you are really deciding to run migrations one by one, then at that point, you need to know that d7_node_title_label needs to run to have your node configurations completed. It's still not a migration dependency that it runs before d7_node, d7_node doesn't care. You care, and so you should probably choose to run it prior to inspecting your content type. But people configuring their content types manually or in some custom way shouldn't have to be stuck on a 'dependent' migration that they don't care about and then have to go figure out that it's not an actual dependency and have to go remove it.

wim leers’s picture

Regardless of which order you run the migrations, after running both your system is in the same state.

Only if you run the entire suite of migrations.

They should be able to pick and choose what they want to migrate, and not have to deconstruct a complex and arbitrary dependency graph to figure what dependencies are actually required, and which are there just because we feel a little better about the order they run.

This is hardly "arbitrary".

This is not possible to do in core. There is no mechanism to pick and choose migrations to run

With drush it's totally possible.

Note I only provided screenshots for "inspecting the data model", but there's more: Node entity validation errors (which is possible in Migrate since #2745797: Add option to content entity destinations for validation) involving the title field would show the wrong title label. What's Name or Foobar in D7 would be Title in D8. The mismatch would be confusing.

quietone’s picture

Honestly, I was a little unsure about this but made that patch anyway hoping for some feedback. Both heddn and mikelutz raise good points but I think that mikelutz is correct that this isn't a migration dependency. I've got nothing to add as mikelutz said it all.

But people configuring their content types manually or in some custom way shouldn't have to be stuck on a 'dependent' migration that they don't care about and then have to go figure out that it's not an actual dependency and have to go remove it

Yes! I think this will cause grief and confusion, let's avoid that.

mikelutz’s picture

Status: Needs review » Closed (won't fix)
webchick’s picture

Version: 8.9.x-dev » 9.0.x-dev
StatusFileSize
new8.2 KB

Uploading a version of this that works for 9.0.x. The hunk for LegacyMigrateUrlAliasTest.php fails because that doesn't exist anymore.