I have a Drupal 6 site with 7 user profile fields grouped into 2 categories. I have 874 users on the site, all with data in the 7 profile fields. When I migrate the site (using drush migrate-import --all) I do not get any errors related to user. All the 874 users are migrated to the Drupal 8 site. All 7 user profile fields are created (but not grouped into categories - that's fine), however no values are populated into the user profile fields. I expect all the values from Drupal 6 user profile fields to be populated into the newly created Drupal 8 user fields.

Output from mysql:
mysql> select * from user__profile_country;
Empty set (0.09 sec)

mysql> select * from user__profile_firstname;
Empty set (0.10 sec)

mysql> select * from user__profile_surname;
Empty set (0.10 sec)

mysql> select * from user__profile_birthyear;
Empty set (0.10 sec)

The tables: users, users_data and users_field_data are populated.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Matt B created an issue. See original summary.

Matt B’s picture

Issue summary: View changes
Matt B’s picture

This is the migrate_plus.migration.upgrade_d6_user.yml file, generated by the migrate process. I notice none of the profile fields are referenced, should they be?

uuid: 875600d9-40e1-46e5-8fd5-1c87e9f001d0
langcode: en
status: true
dependencies: {  }
id: upgrade_d6_user
migration_tags:
  - 'Drupal 6'
migration_group: migrate_drupal_6
label: 'User accounts'
source:
  plugin: d6_user
process:
  uid: uid
  name: name
  pass: pass
  mail: mail
  created: created
  access: access
  login: login
  status: status
  timezone:
    plugin: user_update_7002
    source: timezone
  preferred_langcode: language
  init: init
  roles:
    plugin: migration
    migration: upgrade_d6_user_role
    source: roles
  user_picture:
    plugin: migration
    migration: upgrade_d6_user_picture_file
    source: uid
    no_stub: true
destination:
  plugin: 'entity:user'
  md5_passwords: true
migration_dependencies:
  required:
    - upgrade_d6_user_role
  optional:
    - upgrade_d6_user_picture_file
    - upgrade_user_picture_entity_display
    - upgrade_user_picture_entity_form_display
    - upgrade_d6_user_role
mikeryan’s picture

Status: Active » Postponed (maintainer needs more info)

Profile field values are migrated in d6_profile_values, not in d6_user - what does upgrade_d6_profile_values.yml look like?

Matt B’s picture

Hi Mike, see below:

uuid: e265f5ad-4c1e-4268-89fb-c56c74101985
langcode: en
status: true
dependencies: {  }
id: upgrade_d6_profile_values
migration_tags:
  - 'Drupal 6'
migration_group: migrate_drupal_6
label: 'Profile values'
source:
  plugin: d6_profile_field_values
process:
  uid: uid
destination:
  plugin: 'entity:user'
migration_dependencies:
  required:
    - upgrade_d6_user
    - upgrade_user_profile_field_instance
    - upgrade_user_profile_entity_display
    - upgrade_user_profile_entity_form_display
  optional: {  }

I'm using migrate_upgrade 8.x-3.0-rc1 migrate_plus 8.x-3.0-beta1 and migrate_tools 8.x-3.0-beta1.

Matt B’s picture

Status: Postponed (maintainer needs more info) » Active
mikeryan’s picture

Status: Active » Postponed (maintainer needs more info)

Did the profile values migration generate any errors?

drush mmsg upgrade_d6_profile_values
Matt B’s picture

Status: Postponed (maintainer needs more info) » Active

No messages for this migration [status]

Matt B’s picture

Version: 8.2.1 » 8.2.2

Just upgraded to Drupal 8.2.2 and tested again - confirmed this is still an issue.

albertski’s picture

I have the same issue with profile values not migrating over. When I run command: drush mmsg upgrade_d6_profile_values I get:

drush mmsg upgrade_d6_profile_values
[status] No messages for this migration

albertski’s picture

I am trying to debug this and not sure if this is the issue but in EntityContentBase::updateEntity() it does a foreach of $row->getDestination(). Problem is that $row->destination only contains uid. array('uid' => xx). I'm guessing that destination should have all the fields. (Example first_name, last_name, etc..)

albertski’s picture

Status: Active » Needs review
FileSize
937 bytes

I was able to get this to work properly by setting the destination property of all fields.

albertski’s picture

FileSize
1.24 KB
mikeryan’s picture

Assigned: Unassigned » mikeryan
mikeryan’s picture

Project: Drupal core » Migrate Upgrade
Version: 8.2.2 » 8.x-3.x-dev
Component: migration system » Code
Status: Needs review » Active

I tested locally, running migration from a D6 site with a few profile fields to D8. Doing a straight migrate-upgrade, it worked - the values were properly migrated. However, doing migrate-upgrade --configure-only followed by migrate-import --all, the values did not get migrated. So, it seems like the --configure-only is missing something in setting up the migration configuration entities that breaks the profile value migration.

drzraf’s picture

This comment enlightened me.
Indeed I encounter different kind of bugs between
* migrate-upgrade --configure-only + migrate-import --all
and
* migrate-upgrade alone

I'm also under the impression that calls to drush ms and drush cc drush can affect the way a migration takes place.

mikeryan’s picture

Yes, one distinct possibility is the problem is with the dual-plugin-manager situation being addressed in #2795447: Use the core plugin manager/#2830036: MigrationPluginManager::getDefinitions() blows up in node derivers/#2752335: Properly integrate configuration-entity-based migrations with the core plugin manager. The next thing to do (I probably won't have time today) would be to test with all those patches.

drzraf’s picture

I'd rather not want to pollute this specific issue, but just to add to this:

  • migrate-manifest
  • migrate-upgrade --configure-only + migrate-import --all
  • migrate-upgrade
  • UI

behave differently.

I found that migrate-manifest and UI often give best results.
--configure-only creates strange issues (I personally experience such issues with {d6_term_node} and {d6_field_instance_widget_settings})
On the contrary, UI cause trouble with the {d6_file} migration (NULL destid1)

Diff'ing DB dump of the migrate_map* tables from two instances migrated using distinct methods makes the issue clear: destid* columns are NULL when using migrate-upgrade.

mikeryan’s picture

Status: Active » Postponed (maintainer needs more info)

Per my theory in #17 - can anyone reproduce this problem with the migrate_plus and migrate_tools 8.x-4.0-beta1 releases?

Matt B’s picture

Status: Postponed (maintainer needs more info) » Active

I have now retested using Drupal 8.3.2, migrate upgrade 8.x-3.0-rc1, migrate_plus 8.x-4.0-beta1 and migrate_tools 8.x-4.0-beta1, and I am still not getting the profile fields populated. They are created, but contain no data.

uuid: ef3d4d3b-2293-4971-98bf-fbc71d6b5274
langcode: en
status: true
dependencies: {  }
id: upgrade_d6_profile_values
class: null
field_plugin_method: null
cck_plugin_method: null
migration_tags:
  - 'Drupal 6'
migration_group: migrate_drupal_6
label: 'Profile values'
source:
  plugin: d6_profile_field_values
process:
  uid: uid
destination:
  plugin: 'entity:user'
migration_dependencies:
  required:
    - upgrade_d6_user
    - upgrade_user_profile_field_instance
    - upgrade_user_profile_entity_display
    - upgrade_user_profile_entity_form_display
  optional: {  }

No messages from 'drush mmsg upgrade_d6_profile_values'

heddn’s picture

Status: Active » Postponed (maintainer needs more info)

Looks like we have one report of this not working, are there any other reports of this working or not with the latest versions of things?

heddn’s picture

Status: Postponed (maintainer needs more info) » Closed (cannot reproduce)

Cleaning up the issue queue. If we can reproduce this with the latest code, then please re-open with complete steps on how to reproduce this issue.

anairamzap’s picture

Hi, I'm trying to migrate profile values from D6 to D8 and I'm finding some problems to migrate a picture field.
In D6 I have two profile "fields":

Title          Name	                      Type
Photo        field_profile_image	picture
TestText      profile_test_text  	        textfield

When I run drush mi upgrade_d6_profile_values -execute-dependencies

The textfield field, correctly imports into D8 (it creates the field instance and imports the value), but for the picture field I get:
Attempt to create a field field_profile_image that does not exist on entity type user.

If I manually create the field in D8, I no longer get the error, but the values are not imported.

I'm not sure if this is something to do with missing dependencies or if its related to the field type.
In the upgrade_user_profile_field.yml file generated by drush profile-upgrade --configure-only I see

process:
  entity_type: constants/entity_type
  field_name: name
  type:
    plugin: static_map
    source: type
    map:
      checkbox: boolean
      date: datetime
      list: text
      selection: list_string
      textfield: text
      textarea: text_long
      url: link

That is, the only missing type map is picture type (not sure if this is the expected). Shouldn't be a map "image: picture" in that process section?

Attaching here the relevant yml files.

  • Drupal: 8.4.3
  • migrate_tools: 8.4.x-dev
  • migrate_upgrade: 8.3.x-dev
  • migrate_plus: 8.4.x-dev

Let me know if this should be a new issue and I will create it, but I thought it was related to the original post.

Cheers!

m

Olarin’s picture

I encountered this issue (no profile field values getting saved) while working on https://www.drupal.org/project/drupal/issues/2797047, and have included there a modified version of the fix in #13. (using migrate-upgrade --configure-only and then piecemeal migrate-import)

cestmoi’s picture

I have the latest versions as below and I am having the same issue of the original post here. Profile fields are created but no values are migrated. Migration still shows that all values were migrated ! drush mmsg shows nothing.

All codes are up to date:
Drupal version : 8.5.5
migrate_manifest : 8.x-1.7
migrate_plus : 8.x-4.0-dev
migrate_tools: 8.x-4.x-dev
migrate_upgrade: 8.x-3.x-dev

I first did the migrate-upgrade --configure-only then used migrate-import for users . I had to add "no_stub: true" to roles in the upgrade_d6_user config yml to make the users migration work. Otherwise it would stop at 12 users no matter what. I have over 6000 users. Roles were not migrated. All roles ignored (I have just the Adminstrator and an "Editor" roles).

drush migrate:manifest --legacy-db-url=mysql://root:@localhost/ drupal_6_manifest_users.yml
		 [ok] Running upgrade_d6_filter_format
		 [ok] Running upgrade_d6_user_role
		 [ok] Running upgrade_d6_user
		 [ok] Running upgrade_d6_user_settings
		 [ok] Running upgrade_d6_user_mail
		 [ok] Running upgrade_user_profile_field
		 [ok] Running upgrade_user_profile_field_instance
		 [ok] Running upgrade_user_profile_entity_display
		 [ok] Running upgrade_user_profile_entity_form_display
		 [ok] Running upgrade_d6_profile_values
		 [ok] Running upgrade_d6_user_contact_settings

No roles were migrated still.

drush ms upgrade_d6_profile_values
 ----------------------------------------- --------------------------- -------- ------- ---------- ------------- ---------------
  Group                                     Migration ID                Status   Total   Imported   Unprocessed   
 ----------------------------------------- --------------------------- -------- ------- ---------- ------------- ---------------
  Import from Drupal 6 (migrate_drupal_6)   upgrade_d6_profile_values   Idle     6198    6197       0

I did rollback

drush mr upgrade_d6_profile_values
 [notice] Rolled back 6198 items - done with 'upgrade_d6_profile_values'

I created a "Editor" role manually and added it to two users. I also added the Administrator role to my account.

1. In admin/config/people/accounts/fields
- I could see the profile values fields there but duplicated
- I deleted the extra ones (e.g. profile_gender1, profile_country1 ...etc )
2. On fields Manage Form Display : admin/config/people/accounts/form-display
- The above fields were disabled.
- I enabled them.
3. in Manage Display : admin/config/people/accounts/display
- They're all disabled too.
- I enabled them.

I tried the migration again and got same results:
drush migrate-import upgrade_d6_profile_values

all done .

drush ms upgrade_d6_profile_values

drush ms upgrade_d6_profile_values
 ----------------------------------------- --------------------------- -------- ------- ---------- ------------- ---------------
  Group                                     Migration ID                Status   Total   Imported   Unprocessed   
 ----------------------------------------- --------------------------- -------- ------- ---------- ------------- ---------------
  Import from Drupal 6 (migrate_drupal_6)   upgrade_d6_profile_values   Idle     6198    6197       0

No values were migrated still !

This is my config yml :

uuid: ......
	langcode: en
	status: true
	dependencies: {  }
	id: upgrade_d6_profile_values
	class: Drupal\user\Plugin\migrate\ProfileValues
	field_plugin_method: null
	cck_plugin_method: null
	migration_tags:
	  - 'Drupal 6'
	  - Content
	migration_group: migrate_drupal_6
	label: 'Profile values'
	source:
	  plugin: d6_profile_field_values
	process:
	  uid: uid
	destination:
	  plugin: 'entity:user'
	migration_dependencies:
	  required:
		- upgrade_d6_user
		- upgrade_user_profile_field_instance
		- upgrade_user_profile_entity_display
		- upgrade_user_profile_entity_form_display
	  optional: {  }

This migration experience has been very frustrating and full of issues and I'm glad now the only issue left is this profile_values one and all the rest was done well.

EDIT: I tried to apply @albertski patch (at #13) but it failed to patch (in command line) to my D5.5.5 . I feel it would work for me if the patch was applied.

UPDATE: I applied the above patch manually and did migration rollback on profile_values and repeated the migration and it's working and importing while writing this update as I can see the DB tables updated with users data.

heddn’s picture

There's a test over in #3023504: Add test coverage for drush config export. The reason why this seems to be happening is that we are trying to generate a config entity in core inside of the profile field migration and that is failing:

diff --git a/core/modules/user/src/Plugin/migrate/ProfileValues.php b/core/modules/user/src/Plugin/migrate/ProfileValues.php
index 8399436c03..a36f81ae2a 100644
--- a/core/modules/user/src/Plugin/migrate/ProfileValues.php
+++ b/core/modules/user/src/Plugin/migrate/ProfileValues.php
@@ -45,15 +45,19 @@ public function getProcess() {
             [
               'migration' => 'user_profile_field',
               'source_ids' => $fid,
+              'no_stub' => TRUE,
             ];
           $plugin = $this->processPluginManager->createInstance('migration_lookup', $configuration, $profile_field_migration);
           $new_value = $plugin->transform($fid, $migrate_executable, $row, 'tmp');
+          // Set the destination to the migrated profile field name.
           if (isset($new_value[1])) {
-            // Set the destination to the migrated profile field name.
+            // Use the migrated name.
             $this->process[$new_value[1]] = $name;
           }
           else {
-            throw new MigrateSkipRowException("Can't migrate source field $name.");
+            // Config migrations cannot stub, so just use the name without
+            // transformation.
+            $this->process[$name] = $name;
           }
         }
       }

TheBurt22’s picture

The same issue still persists. I am using
drupal/core 9.1.6
Manually applying the patch from #13 fixes it just fine. Are there plans to apply it to core?

twardnw’s picture

FileSize
1.23 KB

Just ran into this issue as well, I wasn't able to automatically apply the patch in #13 using composer, so I re-rolled against 9.3.16

Anybody’s picture

Assigned: mikeryan » Unassigned

Looking at the patch in #13 I'm asking myself if anyone with deeper Migrate knowledge can tell from the code, if setting the destination plugin is clearly missing. If that's the case, things are quite clear here.

Then we should reroll the patch and get it in.