upgrade_d6_user InvalidArgumentException roles permissions problem [#2892450]

InvalidArgumentException occurs when migrating d6 users. This migration is successful using Drupal 8.2.8.

The problem may be related to upgrade_d6_user_role. 8.2.8 successfully processes all 8 roles; 8.3.5 ignores three roles (1, 2 and 3) and leaves destid1 NULL in the migration map table. There are no messages in the migration message table. The migration fails on the second user.

Processed 8 items (5 created, 0 updated, 0 failed, 3 ignored) - done [status]
with 'upgrade_d6_user_role'
exception 'InvalidArgumentException' with message 'Passed variable is[error]
not an array or object, using empty array instead' in
/Users/tce/Sites/drupal-8.3.5/core/modules/migrate/src/Plugin/migrate/process/Flatten.php:51
Stack trace:
#0
/Users/tce/Sites/drupal-8.3.5/core/modules/migrate/src/Plugin/migrate/process/Flatten.php(51):
ArrayIterator->__construct('')
#1
/Users/tce/Sites/drupal-8.3.5/core/modules/migrate/src/MigrateExecutable.php(380):
Drupal\migrate\Plugin\migrate\process\Flatten->transform('',
Object(Drupal\migrate_tools\MigrateExecutable),
Object(Drupal\migrate\Row), 'permissions')
#2
/Users/tce/Sites/drupal-8.3.5/core/modules/migrate/src/Plugin/migrate/process/MigrationLookup.php(223):
Drupal\migrate\MigrateExecutable->processRow(Object(Drupal\migrate\Row),
Array)
#3
/Users/tce/Sites/drupal-8.3.5/core/modules/migrate/src/MigrateExecutable.php(366):
Drupal\migrate\Plugin\migrate\process\MigrationLookup->transform('8',
Object(Drupal\migrate_tools\MigrateExecutable),
Object(Drupal\migrate\Row), 'roles')
#4
/Users/tce/Sites/drupal-8.3.5/core/modules/migrate/src/MigrateExecutable.php(203):
Drupal\migrate\MigrateExecutable->processRow(Object(Drupal\migrate\Row))
#5 [internal function]: Drupal\migrate\MigrateExecutable->import()
#6
/Users/tce/Developer/packages/drush-8.1.12/includes/drush.inc(720):
call_user_func_array(Array, Array)
#7
/Users/tce/Developer/packages/drush-8.1.12/includes/drush.inc(711):
drush_call_user_func_array(Array, Array)
#8
/Users/tce/Sites/drupal-8.3.5/sites/all/modules/migrate_tools/migrate_tools.drush.inc(282):
drush_op(Array)
#9 [internal function]:
_drush_migrate_tools_execute_migration(Object(Drupal\migrate\Plugin\Migration),
'upgrade_d6_user', Array)
#10
/Users/tce/Sites/drupal-8.3.5/sites/all/modules/migrate_tools/migrate_tools.drush.inc(248):
array_walk(Array, '_drush_migrate_...', Array)
#11 [internal function]: drush_migrate_tools_migrate_import()
#12
/Users/tce/Developer/packages/drush-8.1.12/includes/command.inc(422):
call_user_func_array('drush_migrate_t...', Array)
#13
/Users/tce/Developer/packages/drush-8.1.12/includes/command.inc(231):
_drush_invoke_hooks(Array, Array)
#14 [internal function]: drush_command()
#15
/Users/tce/Developer/packages/drush-8.1.12/includes/command.inc(199):
call_user_func_array('drush_command', Array)
#16
/Users/tce/Developer/packages/drush-8.1.12/lib/Drush/Boot/BaseBoot.php(67):
drush_dispatch(Array)
#17
/Users/tce/Developer/packages/drush-8.1.12/includes/preflight.inc(66):
Drush\Boot\BaseBoot->bootstrap_and_dispatch()
#18 /Users/tce/Developer/packages/drush-8.1.12/drush.php(12):
drush_main()
#19 {main}

I can look into this some more if given suggestions where to look.

Comment	File	Size	Author
#12	specify_migration_id_for_user_role_filter_permission-2892450-12.patch	582 bytes	olarin
#9	fix_default_migration_id_for_d6_filter_format-2892450-9.patch	813 bytes	olarin

Comments

Comment #1

5 July 2017 at 20:30

Tom Elwertowski created an issue. See original summary.

Comment #2

steve hanson commented 5 July 2017 at 21:52

Same here - I am seeing one role import. and all the others are ignored. Then after that there is of course a whole lot of user import breakage.

Comment #3

mikeryan

he/him

English

Pittsfield, MA, USA

commented 6 July 2017 at 20:48

Status:	Active	» Closed (duplicate)
Related issues:		+#2850312: d6_filter_format is hard coded, causing dependent migration to fail

Please see #2850312: d6_filter_format is hard coded, causing dependent migration to fail.

Comment #4

Tom Elwertowski commented 6 July 2017 at 21:42

Status:

Closed (duplicate)

» Active

This may be similar but is not a duplicate. This bug was filed against Drupal 8.3.5 (which includes #2850312: d6_filter_format is hard coded, causing dependent migration to fail).

Comment #5

steve hanson commented 7 July 2017 at 17:27

Yes, my test was against 8.3.5 as well. In 8.3.4 it also failed, but differently.

Comment #6

quietone commented 9 July 2017 at 08:34

Issue tags:

+migrate-d6-d8

Add tag.

Comment #7

Tom Elwertowski commented 13 July 2017 at 03:27

Looking at this some more, I found the cause of the failure. There is a MySQL table name skew.

The upgrade_d6_filter_format migration creates a table named migrate_map_upgrade_d6_filter_format. The upgrade_d6_user_role migration looks for (and creates) a table named migrate_map_d6_filter_format (no upgrade in name). The table is empty so some roles are ignored.

Similarly, the upgrade_d6_user_role migration creates a table with upgrade in the name and the upgrade_d6_user migration looks for (and creates) a table named migrate_map_d6_user_role (no upgrade in name). This one is empty too and results in the exception I reported.

A quick fix to make the user import work is to specify migration: upgrade_d6_filter_format for the filter_format_permission process plugin in upgrade_d6_user_role.yml and add the upgrade_ prefix to the two migration: d6_ specifications in upgrade_d6_user.yml.

I suspect the real fix is to make adding (or not adding) the upgrade_ prefix consistent somewhere in the migration code.

Comment #8

olarin commented 21 July 2017 at 16:06

Priority:

Normal

» Major

I'm having the same issue with 8.3.5. Seems reasonable to me to consider this "Major" priority? It blocks the d6 migration process for many use cases.

Comment #9

olarin commented 23 July 2017 at 19:36

Status:

Active

» Needs review

Status	File	Size
new	fix_default_migration_id_for_d6_filter_format-2892450-9.patch	813 bytes

After further reading of the code and issue #2850312 that mikeryan suggested this duplicates, I think I understand a little better, and it may well be a duplicate - but if so then I think 2850312 should not have been closed. If I'm not totally off base (which I may very well be), I think the default value for the migration ID that's being set in create method in FilterFormatPermission.php should be updated to include the "upgrade_" prefix. I don't know if this is actually the correct approach to the issue, but it does result in user roles importing correctly at least, so I'm posting it as a patch in the hopes of encouraging someone more knowledgeable about the migration system than I to take a look.

Comment #10

23 July 2017 at 20:29

Status:

Needs review

» Needs work

The last submitted patch, 9: fix_default_migration_id_for_d6_filter_format-2892450-9.patch, failed testing. View results

Comment #11

olarin commented 27 July 2017 at 23:17

Ah - after a few more days' digging, now I understand why my patch is silly and should feel bad. d6_filter_format is indeed the correct default for the migration ID. The "upgrade_" prefix comes from using the contrib migrate-upgrade and migrate-tools modules to run the upgrades from Drush (which, to be fair, is kind of the only good way to run migrations, at least when you want to step through piece by piece and see what is and isn't working and why). It still seems reasonable to me to expect all of this to work correctly out of the box, but I think the correct solution is a subtler combination of a patch to core to explicitly specify the default migration id for the FilterFormatPermission process plugin, and a patch to migrate_upgrade to make sure it correctly prefixes such migration ids (it tries to iterate through process plugins but right now I don't think it catches everything).

Comment #12

olarin commented 28 July 2017 at 02:35

Status:

Needs work

» Needs review

Status	File	Size
new	specify_migration_id_for_user_role_filter_permission-2892450-12.patch	582 bytes

Here's a patch which adds the default migration ID for d6_filter_format as an explicit parameter for filter_format_permission in the d6_user_role migration template. By itself I believe it should have no effect, but it should allow the patch I just submitted to Migrate Upgrade in https://www.drupal.org/node/2898028 to properly prefix the ID when running the user role migration via drush migrate-upgrade and migrate-import.

Meanwhile, I think you should find that the similar issues for d6_user have already been solved in the latest dev version of Migrate Upgrade, by the commit for https://www.drupal.org/node/2875405.

Comment #13

28 July 2017 at 03:49

Status:

Needs review

» Needs work

The last submitted patch, 12: specify_migration_id_for_user_role_filter_permission-2892450-12.patch, failed testing. View results

Comment #14

olarin commented 4 August 2017 at 18:50

Status:

Needs work

» Needs review

Comment #15

heddn

English

Illinois

commented 18 September 2017 at 17:25

Status:	Needs review	» Closed (duplicate)
Related issues:		+#2898028: filter_format_permission and d6_cck_file fails on unprefixed migration parameter for filter_format_permission

Duplicate of #2898028: filter_format_permission and d6_cck_file fails on unprefixed migration parameter for filter_format_permission.

upgrade_d6_user InvalidArgumentException roles permissions problem

Comments

Comment #1

Comment #2

Comment #3

Comment #4

Comment #5

Comment #6

Comment #7

Comment #8

Comment #9

Comment #10

Comment #11

Comment #12

Comment #13

Comment #14

Comment #15

Related issues

News items

Our community

Documentation

Drupal code base

Governance of community