Setting this to major because it is a common situation that totally aborts the migration process.

To recreate the problem, create several view modes in the source database. Set some fields to be hidden in some of the view modes. Then attempt a migration. When you get to the migration process for d6_field_formatter_settings, the migration will stop with a message like:

Failed to lookup array (
0 => 'text',
1 => 'hidden',
) in the static map.

I think hidden fields should be treated like excluded fields and just skipped when processing view modes. I'm attaching a patch that would do that. It allowed me to get through the migration without errors when I had hidden fields. And in the resulting D8 site all the fields looked right, hidden in the view modes they should be hidden on.

This was one cause of problems noted in https://www.drupal.org/node/2361401.

Comments

karens’s picture

StatusFileSize
new2.09 KB

Here's the solution I came up with.

karens’s picture

Priority: Major » Normal

Probably not major to Drupal core as a whole, but critical for the migration system.

benjy’s picture

The fix makes sense if the fields default to hidden in D8 which is what the IS says.

Can we add a test for this?

karens’s picture

I'm not sure how the D8 test system works, so this might take a while. If someone wants to add a test to this that would be great :)

karens’s picture

benjy’s picture

Issue tags: +Novice

Tagging as novice, if someone wants to work on this feel free to ping me in #drupal-migrate for some assistance.

th3m0d’s picture

I'm going to jump in on this one. So this fix is good to go if all the fields are hidden @benjy said. What does IS stand for? Trying to determine if you're asking for a conditional check inside the function or a unit test to review the patch.

EDIT: After further reading I understand what context you guys are using the word 'test' in. I should have this done in the next day or so.

ultimike’s picture

Title: Fields hidden in some view modes abort migration » Fields hidden in some view modes abort migration (D6)
mikeryan’s picture

@th3m0d: Did you have a chance to work on a test for this?

Thanks.

neclimdul’s picture

StatusFileSize
new2.05 KB

Not sure if this is the right fix but here is a re-roll around The Great 2015 Migration Migration

neclimdul’s picture

StatusFileSize
new2.01 KB

another reroll

sdstyles’s picture

Status: Active » Needs review

The last submitted patch, 1: 2498291.diff, failed testing.

The last submitted patch, 10: fields_hidden_in_some-2498291-10.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 11: fields_hidden_in_some-2498291-11.patch, failed testing.

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 11: fields_hidden_in_some-2498291-11.patch, failed testing.

neclimdul’s picture

Version: 8.1.x-dev » 8.0.x-dev
Status: Needs work » Needs review

That's weird... I wonder how we got stuck on 8.1.x

quietone’s picture

I can't reproduce the error. But then, the views I made aren't migrated at all. No error messages on screen nor could I find any in log files.

Do you mean select 'exclude from display' when you say "Set some fields to be hidden in some of the view modes"?

karens’s picture

This has nothing to do with views or 'exclude from display'. I'm talking about when you set up the display for a content type and on some view modes, like the 'teaser' view mode, you have some fields hidden. Exclude from display means they are still rendered, hidden means they are not rendered at all.

I just tried the latest bleeding edge branch and this is still a problem. Everything migrates fine until it gets to the formatter migration, where it fails if there are hidden fields, and then at least some of the migrations afterwards all fail. I end up with all my content types and fields set up but no data in them.

neclimdul’s picture

I'll make it a priority to provide test date.

quietone’s picture

@KarenS, thx for the explanation. I thought I had it wrong.

mikeryan’s picture

Status: Needs review » Needs work
Issue tags: +Needs tests

The fix looks good, but we do need to add this scenario to the D6 dump and test it.

Is it an issue in D7 as well?

kari.kaariainen’s picture

This happens in D7: "Missing formatter: The 'text_hidden' formatter used in 1 view modes for the field_myfield field is not available, these displays will be reset to the default formatter."

svendecabooter’s picture

I can't reproduce this in D6 nor D7.
When I set certain fields as hidden in some view modes (full, teaser), I get no errors upon migration to Drupal 8.
On my Drupal 8 install, all fields that have been marked hidden in D6 / D7, are still in the "hidden" region, and thus not rendered on node view.

Could someone who still has this problem elaborate on the steps to reproduce this?

kari.kaariainen’s picture

OK, I now realize that this issue is about hidden fields. I was talking about excluded fields (the excluded column with a checkbox). Sorry.

svendecabooter’s picture

AFAIK Drupal 7 doesn't have that excluded checkbox?
I tested the checkbox in D6, and the result is the same as with a hidden field: in D8 the field gets moved into the Hidden region of the view mode.
So not sure if this issue is still relevant at this point.

kari.kaariainen’s picture

A field that was excluded in D6 doesn't get moved into the Hidden region in D7. Maybe I should add a new issue for that.

quietone’s picture

Issue tags: +migrate-d6-d8

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

mikeryan’s picture

Status: Needs work » Needs review

Can't seem to "add test" on the patch, so forcing a retest (which I'm betting will demonstrate a reroll is needed...)

mikeryan’s picture

StatusFileSize
new2.01 KB

Guess it's too old - re-uploading (committers - no credit for me on this!).

heddn’s picture

Issue tags: -Novice

Looks like we need to add some tests. While that *could* be easy, writing tests isn't generally super simple. Removing the novice tag.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

phenaproxima’s picture

Status: Needs review » Needs work

This definitely needs tests. It shouldn't actually be too tricky -- just load the resulting D8 entity_view_display entity and ensure that the hidden field (or "component" in D8-ese) does not exist in the view display.

mitrpaka’s picture

StatusFileSize
new2.01 KB

Testing if needs re-roll. Some offset found. Patch updated.

curl https://www.drupal.org/files/issues/fields_hidden_in_some-2498291-11_0.patch | patch -p1
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  2061  100  2061    0     0   3366      0 --:--:-- --:--:-- --:--:--  3362
patching file core/modules/field/src/Plugin/migrate/source/d6/FieldInstancePerViewMode.php
Hunk #1 succeeded at 28 (offset -5 lines).
patching file core/modules/node/src/Plugin/migrate/source/d6/ViewMode.php
Hunk #1 succeeded at 21 (offset -5 lines).

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new2.93 KB
new4.95 KB

Changed an existing field to hidden and added a test for that in MigrateFieldFomatterSettings.

The last submitted patch, 39: fields_hidden_in_some-2498291-39-test-only.patch, failed testing.

jofitz’s picture

Status: Needs review » Needs work
Issue tags: -Needs tests +Needs reroll

@quietone Is it possible for $field_row['display_settings'][$view_mode]['format'] to be unset? That's the only issue I can see. Otherwise this is ready for RTBC (after it has been re-rolled).

Also removed the "Needs tests" tag because #39 includes them.

jofitz’s picture

Status: Needs work » Needs review
Issue tags: -Needs reroll
StatusFileSize
new4.94 KB

Re-rolled.

phenaproxima’s picture

Looks great. Nice and simple, and there is test coverage. I just have two tiny concerns:

  1. +++ b/core/modules/field/tests/src/Kernel/Migrate/d6/MigrateFieldFormatterSettingsTest.php
    @@ -21,6 +21,19 @@ protected function setUp() {
    +    $this->assertTrue(is_null($component));
    

    I'd prefer to use assertNull here.

  2. +++ b/core/modules/node/src/Plugin/migrate/source/d6/ViewMode.php
    @@ -21,7 +21,7 @@ protected function initializeIterator() {
    +        if (isset($field_row['display_settings'][$view_mode]) && $field_row['display_settings'][$view_mode]['format'] != 'hidden'  && empty($field_row['display_settings'][$view_mode]['exclude'])) {
    

    This code is repeated in FieldInstancePerViewMode. If one of these classes extends the other, can this logic be a helper method, with a doc comment to explain exactly what it does? Also, a nitpick: there is an extra space before the final &&.

jofitz’s picture

@phenaproxima can you give more details on how you see the helper method working, please? I've got plenty of ideas, but they all seem to have flaws. e.g. FieldInstancePerViewMode could extend ViewMode, but then we are adding calculateDependencies() to FieldInstancePerViewMode. Another idea was to transfer many of ViewMode's methods to ViewModeBase, but I suspect that is not a good idea even though these are the only two classes that extend that.

phenaproxima’s picture

Oh -- I only meant that we should have a helper method if one of the classes already extends the other. If that is not the case, then I'm OK with duplicating the code, as long as there is a comment that is a) duplicated in both places; b) explaining what the if statement is doing; and c) has a @see that mentions the other place where the same if check exists.

jofitz’s picture

StatusFileSize
new3.08 KB
new5.41 KB
  1. Used assertNull()
  2. Added comment to if().
phenaproxima’s picture

Status: Needs review » Reviewed & tested by the community

Excellent! Full stream ahead...

joelpittet’s picture

My other issue seems to conflict with this one where I went with !empty() @Jo Fitzgerald could you review and feel free to close mine if the change isn't needed with what you've changed here.
#2854314: D6 FieldInstancePerViewMode has a default viewmode with no settings

jofitz’s picture

@joelpittet I've taken a look at your ticket, but I don't think there is any conflict with this one. Have a look at my patch(es) for more details on my thoughts: https://www.drupal.org/node/2854314#comment-11994882

joelpittet’s picture

They were just failing to apply with composer patches between each other, didn't diff the diffs just wanted to show they were touching the same lines with empty check

alexpott’s picture

Status: Reviewed & tested by the community » Fixed

Committed and pushed acc5c66 to 8.4.x and 3610a36 to 8.3.x. Thanks!

  • alexpott committed acc5c66 on 8.4.x
    Issue #2498291 by Jo Fitzgerald, neclimdul, quietone, KarenS, mikeryan,...

  • alexpott committed 3610a36 on 8.3.x
    Issue #2498291 by Jo Fitzgerald, neclimdul, quietone, KarenS, mikeryan,...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.