User picture file could not be copied

Use correct parent issue

On #drupal-migrate, weal explained that files are often a bit of an issue because drush doesn't have a parameter for the files location so it often fails. And then suggested using the migrate_upgrade module because it allows you to input a path in the ui at the upgrade. I've not tested that.

Sure. There are new errors this time.

$ sudo -u www-data drush migrate-manifest --legacy-db-url=mysql://root:migrateroot@forest/md6 D6Manifest-User_0.yml
Running d6_filter_format                                                                         [ok]
Running d6_user_profile_entity_display                                                           [ok]
Invalid argument supplied for foreach() RequirementsException.php:63                             [warning]
Migration d6_user_profile_entity_display did not meet the requirements. Missing source provider  [error]
profile 
Running d6_user_profile_entity_form_display                                                      [ok]
Invalid argument supplied for foreach() RequirementsException.php:63                             [warning]
Migration d6_user_profile_entity_form_display did not meet the requirements. Missing source      [error]
provider profile 
Running d6_user_profile_field                                                                    [ok]
Invalid argument supplied for foreach() RequirementsException.php:63                             [warning]
Migration d6_user_profile_field did not meet the requirements. Missing source provider profile   [error]
Running d6_user_profile_field_instance                                                           [ok]
Invalid argument supplied for foreach() RequirementsException.php:63                             [warning]
Migration d6_user_profile_field_instance did not meet the requirements. Missing source provider  [error]
profile 
Running d6_user_role                                                                             [ok]
Running d6_user_picture_field                                                                    [ok]
Running d6_user_picture_field_instance                                                           [ok]
Running d6_user_picture_entity_display                                                           [ok]
Running d6_user_picture_entity_form_display                                                      [ok]
Running d6_user_picture_file                                                                     [ok]
copy(sites/default/files/pictures/picture-7.png): failed to open stream: No such file or         [warning]
directory EntityFile.php:76
File <em class="placeholder">sites/default/files/pictures/picture-7.png</em> could not be copied [error]
to <em class="placeholder">public://pictures/picture-7.png</em>.
copy(sites/default/files/pictures/picture-11.jpg): failed to open stream: No such file or        [warning]
directory EntityFile.php:76
File <em class="placeholder">sites/default/files/pictures/picture-11.jpg</em> could not be copied[error]
to <em class="placeholder">public://pictures/picture-11.jpg</em>.
Running d6_user                                                                                  [ok]
The following migrations were not found: d6_profile_values:user                                  [warning]

And www-data does have read access to the picture file.

$ sudo -u www-data ls -la /opt/md6/sites/default/files/pictures/picture-7.png
-rw-rw-r-- 1 www-data www-data 8456 Jan 18 14:39 /opt/md6/sites/default/files/pictures/picture-7.png

I used D6Manifest-User.yml at Manual testing (D6->D8): User and User Profile

Thanks. But still no luck, adding in the change from #7, I get the following errors:

$ sudo -u  www-data drush migrate-manifest --legacy-db-url=mysql://root:improot@forest/impd6 D6Manifest-User.yml
strpos() expects parameter 1 to be string, array given MigrationStorage.php:30                                              [warning]
substr() expects parameter 1 to be string, array given MigrationStorage.php:31                                          [warning]
substr() expects parameter 1 to be string, array given MigrationStorage.php:34                                             [warning]
array_flip(): Can only flip STRING and INTEGER values! MigrationStorage.php:47                                                  [warning]
array_flip(): Can only flip STRING and INTEGER values! EntityStorageBase.php:233                                      [warning]
PHP Fatal error:  Call to a member function getLoadPlugin() on a non-object in /home/opt/impd8/drupal/core/modules/migrate_drupal/src/MigrationStorage.php on line 73
PHP Stack trace:
PHP   1. {main}() /opt/drush/drush.php:0
PHP   2. drush_main() /opt/drush/drush.php:16
PHP   3. Drush\Boot\DrupalBoot->bootstrap_and_dispatch() /opt/drush/drush.php:76
PHP   4. drush_dispatch() /opt/drush/lib/Drush/Boot/DrupalBoot.php:46
PHP   5. call_user_func_array:{/opt/drush/includes/command.inc:178}() /opt/drush/includes/command.inc:178
PHP   6. drush_command() /opt/drush/includes/command.inc:178
PHP   7. _drush_invoke_hooks() /opt/drush/includes/command.inc:210
PHP   8. call_user_func_array:{/opt/drush/includes/command.inc:359}() /opt/drush/includes/command.inc:359
PHP   9. drush_migrate_manifest() /opt/drush/includes/command.inc:359
PHP  10. entity_load_multiple() /opt/drush/commands/core/migrate.d8.drush.inc:48
PHP  11. Drupal\migrate_drupal\MigrationStorage->loadMultiple() /home/opt/impd8/drupal/core/includes/entity.inc:171
Drush command terminated abnormally due to an unrecoverable error.                                                                                                                                      [error]
Error: Call to a member function getLoadPlugin() on a non-object in /home/opt/impd8/drupal/core/modules/migrate_drupal/src/MigrationStorage.php, line 73

And, again with the original, outdated, manifest:

v@imp {/home/opt/impd8/drupal}$ sudo -u  www-data drush migrate-manifest --legacy-db-url=mysql://root:improot@forest/impd6 D6Manifest-User.yml
Running d6_filter_format                                         [ok]
Running d6_user_role                                             [ok]
Running d6_user_picture_field                                [ok]
Running d6_user_picture_field_instance                    [ok]
Running d6_user_picture_entity_display                   [ok]
Running d6_user_picture_entity_form_display          [ok]
Running d6_user_picture_file                                  [ok]
filesize(): stat failed for public://pictures/picture-1.png File.php:198                                                                                                                 [warning]
filesize(): stat failed for public://pictures/picture-3.jpg File.php:198                                                                                                                  [warning]
Running d6_user                                                                                                            [ok]
filesize(): stat failed for public://pictures/picture-3.jpg File.php:198                                                                                                                  [warning]
Running d6_user_profile_field        [ok]
Running d6_user_profile_field_instance      [ok]
Running d6_user_profile_entity_display     [ok]
Running d6_user_profile_entity_form_display         [ok]
Running d6_profile_values:user    [ok]

I've fixed some configuration problems stemming from my SSD crash and migrations are much improved.

I tried a migration with one user with a profile picture. The user is created but no picture. I found the picture in /tmp/mypix2.png and the following in file_managed.

mysql> select fid,uid,filename,uri from file_managed;
+-----+------+-----------------+-----------------------------------+
| fid | uid  | filename        | uri                               |
+-----+------+-----------------+-----------------------------------+
|   3 | NULL | mypix2.png      | temporary://mypix2.png            |
+-----+------+-----------------+-----------------------------------+
2 rows in set (0.00 sec)

Applied 2417975-1.patch and tried again. No files in D8 and none in /tmp. On the good side, the only errros are the "could not be copied to" for each file.

I had a typo in the manifest file so ignore #11.

Applied 2417975-1.patch and tried again. Profile pix successfully copied to public://pictures/picture-4.png but uid is still NULL.

Ran again today, without 417975-1.patch and with the attached patch. It works, the picture is in the right destination and associated with the user. However, the destination directory, "sites/defaults/pictures", and contents, are chown "myuser:myuser" even though drush was run as user www-data.

But perhaps, this isn't needed if D6->D8 Upload Field Migrations not working is fixed.

OK ran again without the debugger and permissions are as expected.

drwxrwxr-x 2 www-data www-data 4096 Feb 12 17:53 pictures

drwxrwxr-x 2 www-data www-data 4096 Feb 12 17:53 .
drwxrwxrwx 8 v        www-data 4096 Feb 12 17:55 ..
-rw-r--r-- 1 www-data www-data 1654 Feb 12 17:53 picture-3.jpg
-rw-r--r-- 1 www-data www-data 2716 Feb 12 17:53 picture-4.jpg

2369401-17-newtest.patch has just the new tests and should fail and the other patch has the test and the fix.