Problem/Motivation

When running a migration from Drupal 6 to Drupal 8 beta 2 with the latest 7-dev Drush I got an exception when filefield is reached:

exception 'Drupal\migrate\MigrateException' with message 'Failed to [error]
lookup array (
0 => 'filefield',
1 => 'thumb_default',
) in the static map.' in
/var/www/vhosts/d8/core/modules/migrate_drupal/src/Plugin/migrate/process/d6/FieldTypeDefaults.php:33
Stack trace:
#0
/var/www/vhosts/d8/core/modules/migrate/src/MigrateExecutable.php(398):
Drupal\migrate_drupal\Plugin\migrate\process\d6\FieldTypeDefaults->transform(Array,
Object(Drupal

etc.

FileField Version is 6.x-3.10.
Any more information I can provide?

Thanks and best regards,
Rainer for proxiss

Proposed resolution

To skip the row rather than fatal error when the lookup in the static map misses for the field type.

Remaining tasks

Create a patch.

CommentFileSizeAuthor
#45 skip_the_row_when_we-2361401-45.patch2.9 KBsvendecabooter
#41 skip_the_row_when_we-2361401-41.patch2.81 KBneclimdul
#41 2361401-41.interdiff.txt1.06 KBneclimdul
#40 2361401-39.interdiff.txt1.74 KBneclimdul
#39 2361401-39.interdiff.txt0 bytesneclimdul
#39 skip_the_row_when_we-2361401-39.patch2.68 KBneclimdul
#37 skip_the_row_when_we-2361401-36.patch963 bytesneclimdul
#32 2361401-32-migrate-skip-row.patch1008 bytesmikeker
#15 2361401-15.patch999 bytesmikeker
#14 2361401-14-8.interdiff.txt618 bytesmikeker
#14 2361401-14.patch1.07 KBmikeker
#8 2361401-8.patch997 bytesbenjy
#4 2361401-just_for_kicks.diff666 bytesultimike
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

proxiss’s picture

proxiss CreditAttribution: proxiss commented

Ok, i tried some things now:

Starting over with a blank install in english.
=> Import fails like before

Running d6_field_formatter_settings [ok]
exception 'Drupal\migrate\MigrateException' with message 'Failed to lookup array ( [error]
0 => 'filefield',
1 => 'thumb_default',
) in the static map.' in
/var/www/vhosts/d8/core/modules/migrate_drupal/src/Plugin/migrate/process/d6/FieldTypeDefaults.php:33
Stack trace:
#0 /var/www/vhosts/d8/core/modules/migrate/src/MigrateExecutable.php(398):
Drupal\migrate_drupal\Plugin\migrate\process\d6\FieldTypeDefaults->transform(Array,
Object(Drupal\migrate\MigrateExecutable), Object(Drupal\migrate\Row), 'options/type')
#1 /var/www/vhosts/d8/core/modules/migrate/src/MigrateExecutable.php(275):
Drupal\migrate\MigrateExecutable->processRow(Object(Drupal\migrate\Row))
#2 /home/rainer/.composer/vendor/drush/drush/commands/core/migrate.d8.drush.inc(51):
Drupal\migrate\MigrateExecutable->import()

Then I removed to filefield from my Drupal 6 site and resetted everything. Next run:

Running d6_node_setting_promote [ok]
PHP Fatal error: Call to a member function getConfigDependencyName() on a non-object in /var/www/vhosts/d8/core/lib/Drupal/Core/Field/FieldConfigBase.php on line 236
Drush command terminated abnormally due to an unrecoverable error. [error]
Error: Call to a member function getConfigDependencyName() on a non-object in
/var/www/vhosts/d8/core/lib/Drupal/Core/Field/FieldConfigBase.php, line 236

I removed d6_field_formatter_settings from manifest.yml.
Next try got this:

Running d6_field_formatter_settings [ok]
PHP Fatal error: Call to a member function getConfigDependencyName() on a non-object in /var/www/vhosts/d8/core/lib/Drupal/Core/Entity/EntityDisplayBase.php on line 165
Drush command terminated abnormally due to an unrecoverable error. [error]
Error: Call to a member function getConfigDependencyName() on a non-object in
/var/www/vhosts/d8/core/lib/Drupal/Core/Entity/EntityDisplayBase.php, line 165

I removed d6_vocabulary_field_instance
Next try this:

Running d6_vocabulary_field_instance [ok]
PHP Fatal error: Call to a member function getConfigDependencyName() on a non-object in /var/www/vhosts/d8/core/lib/Drupal/Core/Field/FieldConfigBase.php on line 236
Drush command terminated abnormally due to an unrecoverable error. [error]
Error: Call to a member function getConfigDependencyName() on a non-object in
/var/www/vhosts/d8/core/lib/Drupal/Core/Field/FieldConfigBase.php, line 236

Then it run through with tons of warnings and errors and ended up like this:

Migration d6_upload did not meet the requirements. Missing migrations d6_field_formatter_settings, [error]
d6_node. requirements: d6_field_formatter_settings. requirements: d6_node.
The following migrations were not found: d6_field_settings, d6_node_type_entity_display, [warning]
d6_node_type_entity_display_teaser, d6_node_type_entity_form_display, d6_format_settings,
d6_node_body_instance

Users were migrated, but no roles. No Content, no content types, no taxonomies. Blocks were migrated but completely wrong. No books were migrated, no contact forms, no menus etc.

This is really pain in the ass and the hell of an alpha state. I'll try again in some weeks.

Regards,
Rainer

svendecabooter’s picture

svendecabooter
Primary language Dutch
Location Gent
CreditAttribution: svendecabooter commented

I get the same error:
Fatal error: Call to a member function getConfigDependencyName() on a non-object in drupal/core/lib/Drupal/Core/Entity/EntityDisplayBase.php on line 165

It seems this is being worked on in issue #2346039: Add missing migrations to MigrateDrupal6Test and fix the result, so you might follow along there.

ultimike’s picture

ultimike
Primary language English
Location Florida, USA
CreditAttribution: ultimike commented

Hmm - I'm getting hit with this issue as well. Here's the relevant part of my drush migrate-manifest output.

Running d6_field_formatter_settings                                                                          [ok]
exception 'Drupal\migrate\MigrateException' with message 'Failed to lookup array (                           [error]
  0 => 'filefield',
  1 => 'hidden',
) in the static map.' in
/Users/michael/Sites/drupal8/core/modules/migrate_drupal/src/Plugin/migrate/process/d6/FieldTypeDefaults.php:33

I think this is related to ImageField (contrib module) display settings. Just for kicks, I added hidden: filefield_default to d6_field_formatter_settings.yml (see attached diff for details) and re-ran the migration, on the next pass, I got the exception:

Running d6_field_formatter_settings                                                                          [ok]
exception 'Drupal\migrate\MigrateException' with message 'Failed to lookup array (                           [error]
  0 => 'filefield',
  1 => '400px_wide_imagelink',
) in the static map.' in
/Users/michael/Sites/drupal8/core/modules/migrate_drupal/src/Plugin/migrate/process/d6/FieldTypeDefaults.php:33

400px_wide_imagelink is a custom image style formatter on my site.

I'm not sure of the best way to handle this - maybe skip the migration when the field formatter type is not found (instead of throwing an exception)?

-mike

ultimike’s picture

ultimike
Primary language English
Location Florida, USA
CreditAttribution: ultimike commented
FileSize
2361401-just_for_kicks.diff666 bytes

Ah drat - forgot the diff in my last comment. Attached here.

-mike

benjy’s picture

benjy CreditAttribution: benjy commented
Title: CCK FileField 6.x-3.10 breaks the Drupal 8Beta2 upgrade » Skip the row when we don't recognize the the field type

As discussed on the call today, we're going to skip the row rather than fatal error when the lookup in the static map misses for the field type.

areke’s picture

areke CreditAttribution: areke commented
Issue summary: View changes
Tom Elwertowski’s picture

Tom Elwertowski CreditAttribution: Tom Elwertowski commented

Here is some info possibly related to #2349701: FieldTypeDefaults, Uncaught PHP Exception which was closed and redirected here.

The following error occurs for me whenever a Content types > Display fields view mode is set to <Hidden>. Once all are unhidden, my migration proceeds.

exception 'Drupal\migrate\MigrateException' with message 'Failed to lookup array (
  0 => 'text',
  1 => 'hidden',
) in the static map.
benjy’s picture

benjy CreditAttribution: benjy commented
Status: Active » Needs review
FileSize
2361401-8.patch997 bytes

How about we just update the exception to skip the row, like the attached.

quietone’s picture

quietone CreditAttribution: quietone commented

Applied patch and passed tests. Simple patch that looks good and doesn't break anything.

Note that I wasn't able to reproduce the original error, but I tested running 8.0.x. I went ahead and tested any way as I'm leaving for a week long conference today.

ultimike’s picture

ultimike
Primary language English
Location Florida, USA
CreditAttribution: ultimike commented

Seems like we need a test for this patch...

-mike

Anonymous’s picture

Anonymous (not verified) CreditAttribution: Anonymous commented
Status: Needs review » Needs work

I've been encountering the same problem with a D6 database which uses imagefield. I tried the patch in #8, and it does not work. I also tried Ultimike's suggestion in #3, and it hides the errors, but does not resolve the issue.

benjy’s picture

benjy CreditAttribution: benjy commented

@italiatina, can you expand on "does not work" in regards to the patch in #8? If you could post your drush output that would be great.

Tom Elwertowski’s picture

Tom Elwertowski CreditAttribution: Tom Elwertowski commented

Using 8.0.0-beta7, I'm still getting the error described in #7.

mikeker’s picture

mikeker CreditAttribution: mikeker commented
Issue tags: +Needs tests
FileSize
2361401-14.patch1.07 KB
2361401-14-8.interdiff.txt618 bytes

If we're throwing a MigrateSkipRowException then we don't need the use for MigrateException. #8 fixed my errors related to a hidden label field, the attached patch just removes the unneeded use line.

I'm happy to add tests, though I'm new to the migrate project and could use some direction. I assume tests would go into MigrateFieldFormatterSettingsTest.php, correct? Where/How does the D6 field info get built? Any pointers would be great. (And I may have completely missed the docs -- sorry, I'm rushing this evening...) Thanks.

mikeker’s picture

mikeker CreditAttribution: mikeker commented
Status: Needs work » Needs review
FileSize
2361401-15.patch999 bytes

Crap. Wrong patch.

Status: Needs review » Needs work

The last submitted patch, 15: 2361401-15.patch, failed testing.

Status: Needs work » Needs review

mikeker queued 15: 2361401-15.patch for re-testing.

The last submitted patch, 4: 2361401-just_for_kicks.diff, failed testing.

Status: Needs review » Needs work

The last submitted patch, 15: 2361401-15.patch, failed testing.

The last submitted patch, 14: 2361401-14.patch, failed testing.

Status: Needs work » Needs review

alanburke queued 15: 2361401-15.patch for re-testing.

Status: Needs review » Needs work

The last submitted patch, 15: 2361401-15.patch, failed testing.

alanburke’s picture

alanburke CreditAttribution: alanburke commented

The patch works, and allows the migrations to continue.
I'm unsure what the testbot is complaining about.
Will try to retest it once more

Status: Needs work » Needs review

alanburke queued 15: 2361401-15.patch for re-testing.

Status: Needs review » Needs work

The last submitted patch, 15: 2361401-15.patch, failed testing.

Status: Needs work » Needs review

alanburke queued 15: 2361401-15.patch for re-testing.

Status: Needs review » Needs work

The last submitted patch, 15: 2361401-15.patch, failed testing.

tim.plunkett’s picture

tim.plunkett
he/him
Primary language English
Location Philadelphia
CreditAttribution: tim.plunkett at Acquia commented
Version: 8.0.0-beta2 » 8.0.x-dev
Status: Needs work » Needs review

Testbot is behaving as expected, because the issue was assigned to beta2

alanburke’s picture

alanburke CreditAttribution: alanburke commented

Thanks Tim!

alanburke queued 15: 2361401-15.patch for re-testing.

Status: Needs review » Needs work

The last submitted patch, 15: 2361401-15.patch, failed testing.

mikeker’s picture

mikeker CreditAttribution: mikeker as a volunteer commented
Status: Needs work » Needs review
FileSize
2361401-32-migrate-skip-row.patch1008 bytes

Rerolled #15.

@tim.plunkett: Thanks for spotting the bogus Drupal version number... I learn something new each day in the issue queue! :)

joelpittet’s picture

joelpittet
Primary language English
Location Vancouver
CreditAttribution: joelpittet commented
Status: Needs review » Postponed

chx mentioned on #drupal-migrate this is postponed on #2487568: Process plugins should not be allowed to skip rows

KarenS’s picture

KarenS CreditAttribution: KarenS commented

One of the problems noted above comes when you have any fields that are hidden in any view mode. This specific problem can be fixed, I created a separate issue for that at https://www.drupal.org/node/2498291.

ghazlewood’s picture

ghazlewood CreditAttribution: ghazlewood commented

I needed the patch in #32 to get my D6-8 migration to work, otherwise nodes, taxonomy* and most of the jobs in the manifest were skipped. I realise this issue has been postponed but it solved the problem for me on a test migration. Cross referencing this with #2221779: [Meta] Manual Testing: Complete D6 > D8 migrations as the patch that got the migration to work.

*Edited as obviously this patch has nothing to do with taxonomy and that is still not working in my migration

neclimdul’s picture

neclimdul
He/Him
CreditAttribution: neclimdul commented
Status: Postponed » Reviewed & tested by the community

Following the Great Migrate Migration. We've been using this patch internally for months and since there are no changes going to RTBC.

This has been good to go for 3 months and the other issue isn't moving so lets have it clean up this rather then blocking it.

neclimdul’s picture

neclimdul
He/Him
CreditAttribution: neclimdul commented
FileSize
skip_the_row_when_we-2361401-36.patch963 bytes
benjy’s picture

benjy CreditAttribution: benjy commented
Status: Reviewed & tested by the community » Needs work

I think we should have some test coverage here.

neclimdul’s picture

neclimdul
He/Him
CreditAttribution: neclimdul commented
Status: Needs work » Reviewed & tested by the community
Issue tags: -Needs tests
FileSize
skip_the_row_when_we-2361401-39.patch2.68 KB
2361401-39.interdiff.txt0 bytes

tests

neclimdul’s picture

neclimdul
He/Him
CreditAttribution: neclimdul commented
FileSize
2361401-39.interdiff.txt1.74 KB

sorry, real interdiff.

neclimdul’s picture

neclimdul
He/Him
CreditAttribution: neclimdul commented
FileSize
2361401-41.interdiff.txt1.06 KB
skip_the_row_when_we-2361401-41.patch2.81 KB

self review:
Can we get coverage annotations on those tests?

Sure!

benjy’s picture

benjy CreditAttribution: benjy commented

Great +1.

I feel like the MigrateSkipRowException needs a reason message to log, make it easier to diagnose why a row is skipped. Maybe a follow-up to discuss?

EDIT: Could we at least add the message to the exception constructor here?

The last submitted patch, 39: skip_the_row_when_we-2361401-39.patch, failed testing.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 41: skip_the_row_when_we-2361401-41.patch, failed testing.

svendecabooter’s picture

svendecabooter
Primary language Dutch
Location Gent
CreditAttribution: svendecabooter as a volunteer commented
Status: Needs work » Needs review
FileSize
skip_the_row_when_we-2361401-45.patch2.9 KB

* There was an error in the namespace of the test class - fixed now.
* MigrateSkipRowException does have a message parameter now - see #2522012: Remove broken idlist handling, replace with more robust exception handling
* A message has been set when MigrateSkipRowException is thrown.

mikeryan’s picture

mikeryan
he/him
Primary language English
Location Murphysboro, IL, USA
CreditAttribution: mikeryan at Acquia commented
Status: Needs review » Closed (won't fix)

Actually, looking closely at this issue, I think no change is necessary since #2559571: Handle MigrateException consistently during import went in. One point that is poorly understood is that MigrateSkipRowException is meant to indicate that a row is being intentionally skipped - that is, the row is logically considered not part of the source to be migrated (use this when it's difficult to represent the logic for exclusion directly in the source query) - MigrateException is the one to throw in error conditions, and as of #2559571: Handle MigrateException consistently during import that is properly caught and handled.