Problem/Motivation

Content of custom fields on Users is not migrating from D7 to D8 on the first pass. It does migrate if Users (and nothing else) is rolled back and re-migrated. This appears to be because the fields and field instances are not migrated until after the users.

Proposed resolution

Ensure fields and field instances are migrated before users.

Remaining tasks

Add d7_field_instance (and consequently d7_field) as a migration dependency of d7_user (as it is for d7_node).

User interface changes

API changes

Data model changes

CommentFileSizeAuthor
#17 d7_user_fields_not_migrating-2674152-17.patch7.86 KBjofitz
#17 interdiff.txt901 bytesjofitz
#15 d7_user_fields_not_migrating-2674152-15.patch7.76 KBjofitz
#10 sshot-field-instance-0.png2.88 KBhussainweb
#9 d7_user_fields_not_migrating-2674152-9.patch7.72 KBjofitz
#2 d7_user_fields_not_migrating-2674152-2.patch455 bytesjofitz

Comments

Jo Fitzgerald created an issue. See original summary.

jofitz’s picture

jofitz
he/him
commented
Status: Active » Needs review
StatusFileSize
new d7_user_fields_not_migrating-2674152-2.patch455 bytes

Add d7_field_instance to the migration dependencies of d7_user.

quietone’s picture

quietone commented
Issue tags: +migrate-d7-d8
benjy’s picture

benjy commented
Issue tags: +Needs tests
Related issues: +#2675470: D7 field instance settings for any fields not on nodes are not migrating

Can we add a couple of tests here please.

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

jofitz’s picture

jofitz
he/him
commented
Version: 8.1.x-dev » 8.1.1

Checked with v8.1.1.

jofitz’s picture

jofitz
he/him
commented
Version: 8.1.1 » 8.1.x-dev

@benjy I'm not sure how to add tests for this seeing as migrations and their ordering is hard-coded in functional tests. I cannot think of a test that will fail without this patch and then pass with it. Any suggestions?

(Also version corrected)

benjy’s picture

benjy commented

Could we add some assertions to \Drupal\Tests\user\Kernel\Migrate\d7\MigrateUserTest for some of the user field data?

jofitz’s picture

jofitz
he/him
commented
Issue tags: -Needs tests
Related issues: +#2673960: Unable to migrate D7 User cck fields, +#2671312: No default value for User langcode when migrating D7 users with no language
StatusFileSize
new d7_user_fields_not_migrating-2674152-9.patch7.72 KB

Added tests to MigrateUserTest.

N.B. Test user language changed from '' (empty string) to 'en' because fields (with a language) cannot be associated with users without a language (see #2671312: No default value for User langcode when migrating D7 users with no language: No default value for User langcode when migrating D7 users with no language).

(now has a lot of code in common with #2673960: Unable to migrate D7 User cck fields, unsurprisingly)

hussainweb’s picture

hussainweb
Primary language English
commented
StatusFileSize
new sshot-field-instance-0.png2.88 KB

I see this issue was created 5 months ago and it might not be working then but I can't get it to reproduce right now. I downloaded the latest 8.2.x, enabled a few basic modules, and migrated from my D7 site which has two fields on user. They did get migrated the first time:

Upgrading d7_dblog_settings
Upgrading d7_filter_format
Upgrading d7_image_settings
Upgrading d7_image_styles
Upgrading d7_node_settings
Upgrading d7_search_settings
Upgrading d7_taxonomy_vocabulary
Upgrading d7_url_alias
Upgrading d7_user_flood
Upgrading d7_user_mail
Upgrading menu_settings
Upgrading search_page
Upgrading taxonomy_settings
Upgrading text_settings
Upgrading update_settings
Upgrading block_content_type
Upgrading d7_taxonomy_term
Upgrading block_content_body_field
Upgrading d7_custom_block
Upgrading d7_user_role
Upgrading d7_block
Upgrading d7_file
Upgrading user_picture_field
Upgrading user_picture_field_instance
Upgrading user_picture_entity_display
Upgrading user_picture_entity_form_display
Upgrading d7_user
Upgrading d7_node_type
Upgrading d7_field
Upgrading d7_comment_type
Upgrading d7_field_instance
Attempt to create a field field_address that does not exist on entity type node.         [error]
(/var/www/bbd8/web/core/modules/field/src/Entity/FieldConfig.php:286)
Upgrading d7_node:article
Upgrading d7_node:business_profile
Upgrading d7_node:classified_ad
Upgrading d7_node:daily_classifieds
Upgrading d7_node:page
Upgrading d7_node:simplenews
Upgrading d7_comment
Upgrading d7_comment_field
Upgrading d7_comment_field_instance
Upgrading d7_comment_entity_display
Upgrading d7_comment_entity_form_display
Upgrading d7_comment_entity_form_display_subject
array_key_exists(): The first argument should be either a string or an integer           [warning]
NestedArray.php:72
array_key_exists(): The first argument should be either a string or an integer           [warning]
NestedArray.php:72
Upgrading d7_view_modes
Upgrading d7_field_formatter_settings
The "ds_taxonomy_separator" plugin does not exist.                                       [error]
(/var/www/bbd8/web/core/lib/Drupal/Component/Plugin/Discovery/DiscoveryTrait.php:52)
The "colorbox" plugin does not exist.                                                    [error]
(/var/www/bbd8/web/core/lib/Drupal/Component/Plugin/Discovery/DiscoveryTrait.php:52)
The "ds_taxonomy_separator" plugin does not exist.                                       [error]
(/var/www/bbd8/web/core/lib/Drupal/Component/Plugin/Discovery/DiscoveryTrait.php:52)
Upgrading d7_field_instance_widget_settings
Upgrading d7_google_analytics_settings
Upgrading d7_google_analytics_user_settings
Upgrading d7_node_revision:article
Upgrading d7_node_revision:business_profile
Upgrading d7_node_revision:classified_ad
Upgrading d7_node_revision:daily_classifieds
Upgrading d7_node_revision:page
Upgrading d7_node_revision:simplenews
Upgrading d7_node_title_label
Upgrading d7_shortcut_set
Upgrading menu
Upgrading menu_links
Upgrading d7_shortcut
Upgrading d7_shortcut_set_users

The field instance settings didn't get migrated but that's handled in another issue - #2675470: D7 field instance settings for any fields not on nodes are not migrating. The tests in here could be useful but I think there are identical tests in many other issues.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Status: Needs review » Needs work

The last submitted patch, 9: d7_user_fields_not_migrating-2674152-9.patch, failed testing.

mikeryan’s picture

mikeryan
he/him
Primary language English
Location Pittsfield, MA, USA
commented
Issue tags: +Needs reroll

@hussainweb - the OP may be a bit ambiguous - the fields and their instances do get created on migration, but after the users are migrated, thus the field *content* is not populated on users. I've reproduced this locally.

jofitz’s picture

jofitz
he/him
commented
Title: D7 User fields not migrating first time » D7 User fields content not migrating first time
Assigned: Unassigned » jofitz
Issue summary: View changes

Clarification of title and summary.

In process of re-rolling.

jofitz’s picture

jofitz
he/him
commented
Assigned: jofitz » Unassigned
Status: Needs work » Needs review
StatusFileSize
new d7_user_fields_not_migrating-2674152-15.patch7.76 KB

Re-rolled.

Status: Needs review » Needs work

The last submitted patch, 15: d7_user_fields_not_migrating-2674152-15.patch, failed testing.

jofitz’s picture

jofitz
he/him
commented
Status: Needs work » Needs review
Issue tags: -Needs reroll
StatusFileSize
new interdiff.txt901 bytes
new d7_user_fields_not_migrating-2674152-17.patch7.86 KB

Resolve test failure.

The last submitted patch, 15: d7_user_fields_not_migrating-2674152-15.patch, failed testing.

mikeryan’s picture

mikeryan
he/him
Primary language English
Location Pittsfield, MA, USA
commented
Status: Needs review » Reviewed & tested by the community

4 files changed, 84 insertions, 6 deletions.

I'm amused by this, given the actual fix is 1 line...

Anyway, looks good to me, RTBC, thanks!

  • catch committed 3d7c530 on 8.3.x 
    Issue #2674152 by Jo Fitzgerald, hussainweb: D7 User fields content not...

  • catch committed 99f6a4e on 8.2.x 
    Issue #2674152 by Jo Fitzgerald, hussainweb: D7 User fields content not...
catch’s picture

catch
he/him
Primary language English
commented
Priority: Normal » Major
Status: Reviewed & tested by the community » Fixed
Issue tags: +Migrate critical

Committed/pushed to 8.3.x and 8.2.x, thanks!

Also retrospectively bumping to major and 'migrate critical' since this would result in data loss for a one-hit migration via the UI.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.