Migration Files for Drupal 7 Users

User signatures could go contrib for d8

Updated against HEAD and postponed until MigrateDrupal7TestBase lands.

Merged #2132221: Variable to config: user.flood [d7] and #2402201: Migration Files for Drupal 7 Roles and Permissions.

There is talk of changing the migrate/user/password stuff over here: https://www.drupal.org/node/1845004#comment-9997759

Might be worth checking out how that will work with this patch.

Updated the patch, and tests. Removed a bunch of stuff relating to mapping filter format permissions to roles; I don't think this will be necessary, because filter formats don't need to be migrated before roles. (Roles will happily save, even if they contain undefined permissions.)

Cleaned up MigrateUserTest and a couple of other things.

Merged #2414599: Migration Files for Drupal 7 User Picture Field.

Fixing minor test breaks caused by the removal of the fake DB driver.

I've omitted the interdiff because the patch in #15 accidentally included a bunch of stuff from 8.0.x that has nothing to do with user migration. Whoops :)

Re-rolled because the patch was seriously out of date. Changes were fairly extensive so I'm skipping the interdiff.

Postponed on #2534042: Move module-specific migration support into the user module - let's get everything moved first (which will require rerolling this patch).

And now for something completely similar (re-roll).

Let's see how this one does. I consolidated a few migrations which were identical for D6 and D7, so there's a good chance this will fail testing.

Moved test files from Migrate/d6 to Migrate.

There were still some calls to loadDumps and references to d6_user_picture...
Let's try again.

Except for MigrateDrupal6Test, these tests pass locally. This is isn't the first time I've had problems because of the MigrateDrupal6Test. What is the value of MigrateDrupal6Test?

Should be fixed now. It was one damn line in d6_user.yml...and one of the more emotionally exhausting bugfixes of my career thus far.

EDIT: Uh...hmm, I screwed up that upload. The ~55KB one is correct.

thx phenaproxima, I looked right at that file a couple of times and missed it. 'Emotionally exhausting' is right!

Before doing a full code review, I've tried running this with a real D7 site using migrate_upgrade. Issues I've found:

1. d7_user_role creates redundant roles - in addition to the default Administrator, Authenticated User, and Anonymous User roles, I end up with administrator, authenticated user, and anonymous user. We need to map those roles to the existing ones rather than recreating them.
2. d7_user itself dies with "exception 'InvalidArgumentException' with message 'Passed variable is not an array or object, using empty array instead' in /Users/mryan/Sites/d8/core/modules/migrate/src/Plugin/migrate/process/Flatten.php:35". This appears to be happening when mapping the roles using the d7_user_role migration, so may be related to #1.

I'm taking a pass at this atm.

OK, on the user role issue, applying the user_update_8002 process plugin as we did for D6 maps the anonymous and authenticated roles appropriately. We still get Administrator plus administrator, but that's not a built-in role - it's arguable either way whether it should be automatically consolidated or not - if it should, that would be a separate issue.

With the user role issue fixed, I next hit "Fatal error: Call to a member function getFileUri() on a non-object in /Users/mryan/Sites/IMP2/core/modules/image/src/Plugin/Field/FieldType/ImageItem.php on line 318" (with some tweaking to the user_picture migration). Commenting out the picture migration for now led to successful user import, so we've still got some work to do on the pictures.

This is blocked by #2555089: d7_filter_format migration mishandles format IDs.

This is also the first of the "Big Four" migrations for D7, so it is absolutely Migrate-critical, and it blocks the Node migration since nodes need an author :)

Here it is, folks...the working (I hope) patch for user migration, pulled from the IMP2 sandbox.

Unblorked.

Fixed assorted sucktitude and WTFs.

+++ b/core/modules/migrate_drupal/src/Tests/d6/MigrateDrupal6Test.php
@@ -147,10 +147,10 @@ class MigrateDrupal6Test extends MigrateFullDrupalTestBase {
-    'd6_user_picture_entity_display',
-    'd6_user_picture_entity_form_display',
-    'd6_user_picture_field_instance',
-    'd6_user_picture_field',
+    'user_picture_entity_display',
+    'user_picture_entity_form_display',
+    'user_picture_field_instance',
+    'user_picture_field',

Shared migrations, that's cool.

Thar be cruft in them thar tests. Let's try again.

Generalized the user_mail migration -- it's identical between D6 and D7 -- and added assertions for it.

Fixed failures resulting from #55.

Good lord. Once again?

And again. :-)

Oops, got a knock on the door and didn't check test results before submitting the previous patch.

Since D6 user_mail tokens do not get converted was committed the user mail migration includes all the user_mail_* variables, which by the way, are different than the ones in D7 and that is why #60 fails. D7 has the same user_mail variables as D6 plus one more, user_mail_cancel_*. There is a nifty page showing the differences but I can't find it tonight.

The attached patch separates the user_mail migrations, so that the D8 mail cancel_confirm fields are filled by the D6 user_mail_delete variables (as in #2551631) and for D7 they are filled with the user_mail_cancel variables (as in patch #60).

No interdiff, as it fails.

Fixing the failure. Interdiff is being weird for me too, so screw it.

Here are the diffs between D6 and D7 files that exist for both, always enlightening.

Meanwhile, ran both a D6 and D7 site through migrate_upgrade with this patch, users look fine in both.

Just reviewing the diffs between D6 and D7, haven't done a full patch review yet:

1. +++ b/core/modules/user/migration_templates/d7_user.yml
   @@ -13,25 +15,17 @@ process:
   -  user_picture:
   

   Since the D7 users.picture was a fid, and we're preserving IDs (including fids), shouldn't we able to do

     user_picture: picture
   

   here?

2. +++ b/core/modules/user/migration_templates/d7_user_mail.yml
   @@ -20,48 +20,20 @@ source:
   -  'status_activated/subject':
   -    plugin: convert_tokens
   -    source: user_mail_status_activated_subject
   ...
   +   'status_activated/subject': user_mail_status_activated_subject
   

   Just to clarify in my mind - tokens changed between D6 and D7, but not between D7 and D8, thus we don't need the conversion here?

Added profile support by generalizing the d6_profile_* migrations (except d6_profile_field_values). The d7_user migration uses a builder to merge profile properties directly into the user object.

Whoops.

Re #2 in comment #69.
From working on "D6 user_mail tokens do not get converted" I do believe that the '!' style tokens are only in D6.

Wat.

Down to the nits...

1. +++ b/core/modules/user/migration_templates/d7_user.yml
   @@ -0,0 +1,31 @@
   +  init: init
   

   I still feel the one-line

     user_picture: picture
   

   is worth adding here rather than doing in a followup.

2. +++ b/core/modules/user/src/Plugin/migrate/builder/d7/User.php
   @@ -0,0 +1,90 @@
   +    try {
   +      $profile_fields = $this->getSourcePlugin('profile_field', $template['source']);
   +      // Ensure that Profile is enabled in the source DB.
   +      $profile_fields->checkRequirements();
   ...
   +    catch (RequirementsException $e) {
   +      // Profile is not enabled in the source DB, so don't do anything.
   +    }
   

   The try only needs to be around the checkRequirements() call.

The rest looks good to me, and manual tests continue to succesfully import users...

1. Added.
2. It needs to be around the whole thing because I don't even want to try to loop through $profile_fields if it fails the requirements check.

It needs to be around the whole thing because I don't even want to try to loop through $profile_fields if it fails the requirements check.

The getSourcePlugin could be move out of the try, but don't bother unless you're rerolling anyway...

RTBC by my account!

1. +abstract class FieldableEntity extends DrupalSqlBase {
   

   This looks like it'll come in handy for other entity migraitons. yay.

2. +++ b/core/modules/user/src/Plugin/migrate/process/ProfileFieldSettings.php
   @@ -13,15 +13,13 @@
   - *   id = "d6_profile_field_settings"
   + *   id = "profile_field_settings"
   

   A huge chunk of the patch is just doing changes like this; that's awesome that we're able to re-use all of this stuff from the D6 migration. :D

3. +++ b/core/modules/user/src/Plugin/migrate/process/ProfileFieldSettings.php
   similarity index 85%
   rename from core/modules/user/src/Plugin/migrate/process/d6/UserUpdate8002.php
   
   rename from core/modules/user/src/Plugin/migrate/process/d6/UserUpdate8002.php
   rename to core/modules/user/src/Plugin/migrate/process/UserUpdate8002.php
   

   Not introduced in this patch, but WTF at that class name? :P

Committed and pushed to 8.0.x. YEAH! 1 down, 3 to go. :)

Oh, neglected to mention my favourite part which is the new email texts. ;)