Problem/Motivation

Sorry for the blunt issue title, I still don't grok exactly what is happening here to denote a more specific problem. It might be that I'm just using the system wrong but in that case the documentation is severely lacking and/or the code paths are fairly ungrokable.

As part of migrating a D6 site to D8 I utilized the d6_user migration that core ships with. I made some minor modifications for our use-case (such as not migrating the text formats for the user signatures but just using a static map) but left it as is almost completely.

However, when executing, the migration did not complete for me. Especially the migration of the user pictures failed. Here's the path down the rabbit hole:

  1. I noticed that something in \Drupal\migrate\Plugin\migrate\destination\EntityFile::import() went wrong. $this->configuration['source_path_property'] was picture but the passed row did not have a corresponding picture property.
  2. That is because \Drupal\migrate\Plugin\migrate\process\Migration::transform() (which performs the d6_user_picture_file migration as part of the d6_user migration) creates a $stub_row which only contains the ID values of the source (here: the source of the
    d6_user_picture_file migration). \Drupal\migrate_drupal\Plugin\migrate\source\d6\UserPictureFile::getIds(), however, simply declares the uid as an ID.
  3. That's why the attached patch adds the picture column there as well.
  4. Even then, though, the $stub_row does not contain the correct values because $source_id_values[$migration_id] is empty.
  5. That in turn is because $value is passed as NULL into Migration::transform() so one would have to fill $this->configuration['source_ids'][$migration_id] for $source_id_values to be set correctly (see lines 88 and following of Migration.php).
  6. Setting source_ids in the parent migration (i.e. in d6_user) does not have any effect, though, because \Drupal\migrate_drupal\Plugin\migrate\process\d6\UserPicture::create does not pass down the passed $configuration into the Migration plugin but instead creates the child configuration (i.e. that of the d6_user_picture_file migration) from scratch.
  7. That is why the patch adds the passing down of source_ids in UserPicture. At that point $source in EntityFile actually gets set to the correct value.
  8. $destination, however, is empty. That is because the processed $stub_row does not contain a uri property.
  9. The $migration (in Migration::transform(), i.e. d6_user_picture_file) actually contains the correct process plugins to generate the correct D8 URI, but Migration::transform() filters out those process plugins, because it only performs those pertaining to the destination ID properties and EntityFile does not declare uri as an ID.
  10. That is why the patch adds the uri property as an ID. With that the migration actually succeeds and users are migrated.
  11. The actual file entities for the user pictures are also migrated, albeit not yet correctly: The uri is processed with the \Drupal\migrate_drupal\Plugin\migrate\process\d6\FileUri. However, both $file_directory_path and $is_public are empty in FileUri::transform() so that the directory path does not get removed correctly and the private scheme is used instead of public.

At this point I would really like some info on what the heck is going on here and if my debugging efforts make even the slightest sense from a Migration expert's perspective before trying to make those properties work as well.

Proposed resolution

???

Remaining tasks

User interface changes

API changes

Comments

benjy’s picture

I didn't fully ready through the issue summary but can you first try with this patch: #2244555: Use copy() directly instead of file_unmanaged_copy().

It's a known issue that file migrations won't work at all without first applying that core patch.

tstoeckler’s picture

Ahh thanks for the pointer. Will try that!

tstoeckler’s picture

Issue summary: View changes

So that issue introduces a MigrateException if the file_prepare_directory() call fails. That makes the migrate system properly skip that row, so that the migration can at least proceed. With that I can at least complete migrations which depend on this one - even though the user pictures are not migrated. The issues described here are still valid though: Both $source is an empty string and $destination is NULL even with the patch applied.

ultimike’s picture

nixou’s picture

Hi, this message just to say that we are actually working on this problem at Drupal Dev Days.
We will update this thread if we have some informations to help.

nicoloye’s picture

Status: Active » Needs review
StatusFileSize
new689 bytes

This patch for migrate_upgrade module should do the trick for the backoffice part.
Done together with @Nixou on drupal dev days.

Also if you need to test such migration through drush and a manifest file. Add those options to your file (trailing slash is important !)

- d6_user_picture_file:
  destination:
    # Final trailing slash is mandatory.
    source_base_path: http://origin.com/

Status: Needs review » Needs work

The last submitted patch, 6: migrate_upgrade-picture-migration-fix-2336281-7.patch, failed testing.

nicoloye’s picture

Status: Needs work » Needs review

As the patch doesn't concern the core we moved it on the module issues here : https://www.drupal.org/node/2471138

The last submitted patch, d6-user-picture-migration-fix.patch, failed testing.

benjy’s picture

@tstoeckler can you confirm you were using the UI when testing this? If that's the case, we should have just moved this issue into the Migrate Upgrade issue queue.

mikeryan’s picture

Status: Needs review » Closed (duplicate)

This was a contrib issue, long since addressed in the duplicate issue #2471138: User's picture are not imported.