Migration Files for Drupal 7 Content

Wrote a very, very basic test of this migration. In the meantime, it's blocked by #2495755: Create MigrateDrupal7TestBase.

Renaming to reflect the closure of #2466795: Migration Files for Drupal 7 Nodes .

Added a basic d7_node migration and a test for it. This migration does NOT migrate any field data -- only the metadata about the nodes and their revisions. Given the unified way in which D7 handles fields, I think it makes sense to migrate field data after the owning entities' metadata.

Annnd a patch would be helpful.

Added unit tests of the d7_node and d7_node_type source plugins.

Merged #2466793: Migration Files for Drupal 7 Node Settings and wrote a test for it.

Improved the tests, particularly the one for node type migration. I'd say this is now truly ready for review and commit.

It would be nice to have a diff between the D6 versions so we can get an idea of how much is copy/paste vs new stuff. Not sure how easy that is?

It's tedious, but not beyond a bit of shell scripting. Now if only I knew how to write proper shell scripts :)

Working on a re-roll

This is an incomplete reroll and left at Needs Work.

phenaproxima suggested on IRC that this needs to wait on other work (sorry don't know the issues numbers).

It's OK to continue re-rolling it; it simply won't be commit-able until migration paths for text formats and fields have landed.

Since this is one of the "big 4" content migrations, escalating to a Migrate critical.

@phenapromixa, ok. I guess I misunderstood part of what you said. I'll continue then.

Reroll.

One of the failures is on the test of the title field label for the migrated node, in NodeTypeTest. That got me looking at the d6 version and that doesn't test for the title field label. So, I added a test and that fails too. But when running a D6->D8 migration, I used migrate_upgrade module, the title field is correctly migrated. Please enlighten me on how to properly test for this.

Looks like that needs a separate migration, because alterations to the title field are overrides of base node fields. The attached patch provides such a migration, along with a test.

Whoa -- somehow the patch got a lot of extra cruft attached to it (my fault, apparently -- see the jump in file size at #13 :). This is re-roll without the cruft.

Fixing the test failures and re-added the unit tests to the patch.

Fixing the test failure. Configuring the node types to pass the test put a lot of stuff into the {variable} table, but due to migrate-db.sh's hashing system, there's not much I can do about that.

This is blocked by #2414651: Migration Files for Drupal 7 Users.

Unblocked!

Just changing the status. Nothing to see here.

As expected.

Re-rolled from the IMP2 sandbox.

Fixing failures and other assorted sadnesses.

Diffed the relevant migration templates and source plugins.

1. +++ b/core/modules/node/migration_templates/d7_node.yml
   @@ -15,44 +15,22 @@ process:
   +  uid:
   ...
   +    migration: d7_user
   

   D6 should do the same thing, if we don't fix it here we need to open an issue for it.

2. +++ b/core/modules/node/migration_templates/d7_node.yml
   @@ -15,44 +15,22 @@ process:
   -    - d6_filter_format
   

   I think I see why the other dependencies are gone, but it seems we have no dependencies on field migrations? And, although nodes themselves don't need filter formats, 99.9% of the time they will have fields that do (maybe fields should have a dependency on d7_filter_format?).

3. +++ b/core/modules/node/migration_templates/d7_node_revision.yml
   @@ -15,30 +15,21 @@ process:
   -  uid: node_uid
   

   As above, should followup with D6 fix.

Not seeing any significant issues here, but I'm not sure I'm comfortable RTBCing this until we can actually test with migrate_upgrade (brokenness elsewhere is preventing that)...

Re-rolled due to Variable.php conflicts, and fixed #47.1 and #47.3.

Fixed a YAML parse error. I never cease to marvel at just how badly a one-line error can hose everything.

Works with migrate_upgrade, RTBC!

Reverted to statically mapping the uid in d7_node and d7_node_revision. The test failures tie directly into #2560637: Improve handling of uid 1 during migration and are outside the scope of this issue.

Testbot seems to be having a tough day today, so I'm pre-emptively re-RTBCing this since it's passed DrupalCI.

Adam walked me through this patch on IRC. The one thing that was weird was switching from uid 1 to uid 2 as the owner of the node in the database dump, but this was done because of #2560637: Improve handling of uid 1 during migration, which has now been escalated to a Migrate critical.

Everything else looks great, and I had the opportunity to see it actually work LIVE on a hangout with Adam and Mike. W00t!

Committed and pushed to 8.0.x. YEAH! :D