Scrambled fields in manage display after 8.3.0 core update

I'm experiencing the same behaviour

In the config yml files it used to be:

third_party_settings:
  ds:
    regions:
      region-name:
        - field-name

After upgrading to 8.3, there also is a region set inside the content part of the yml file:

content:
  fieldname:
    region: region-name

looks like the second piece is always set to 'region: content', even if that field does not need to be in the content region (and even if the content region does not exist)

Interesting, I will check this with swentel and timplunkett.
This possibly needs a new update function.

I think we've run into this as well, any fields that were previously in the "content" region now just end up in the first region of your layout in 8.3, even if your layout actually has a "content" region.

Attached patch doesn't reset the region if the content region also exists in your layout.

Hmm interesting, I'll have a look at it this evening.

I have the same issue.

If I apply the patch in #4 before upgrading, it works. Thx benjy!

Trying to follow this but I just wanted to add that my particular problem doesn't only deal with the content region. I have several regions in my content types and everything is being shifted to the top region.

Having said that, I'd like to know if there is any danger in simply updating the fields back to their original positions? This is a basically a blocking issue for us to upgrade to 8.3. Will another patch reset it again?

There could be two separate issues here, to clarify, the patch I posted solved an issue where after the upgrade, regardless of what we did, we could not place a field in the content region, it would simply jump back to the first region defined in the layout.

We aren't sure if we had issues with the upgrade or whether developers saved and re-exported the entity displays after the upgrade and the bug I described here simply caused fields to be moved to the content region. We only had a small site so we just put the fields back and everything was OK.

I can't reproduce the problem @ChristopheDG has.
There shouldn't be anything changed causing a complete reorder of the fields...
Did you "uninstall" a module and than reinstall it? Or did you run a regular upgrade?

Can people recreate a scenario for me?

We need at least:
1) The entity view mode config before the upgrade
2) The entity view mode config after the upgrade
3) List of layout modules installed + versions (before and after)

#4 works for me before upgrading, Thanks !

I want to help but everyone with problems is completely ignoring my questions. Can *someone* provide me with the requested information? (see https://www.drupal.org/node/2868893#comment-1205105)

I can probably help with that. I do not know how to get the entity mode config though.

I'd also like to make sure we are upgrading in the proper way given the interaction with the layout plugin. So could someone let me know exactly what order I should be doing those steps in?

I recently upgraded from 8.7.2 to 8.3.1 and my DS fields were also scrambled and duplicated. I ended up uninstalling and reinstalling DS, and DS's functionality returned to normal, and then I had to re-do my region placements, field configurations, etc. Not ideal obviously, but at least was able to move past the issue.

Since 8.3.1 I am missing the region drop down under manage display...

Didn't help for me. May be I have a bit old DS to upgrade. Needed all these updates before Drupal would work:

system module :
  8300 -   Add detailed cron logging configuration.
  8301 -   Add install profile to core.extension configuration.

block_content module :
  8300 -   Fix the block_content entity type to specify its revision data table.

comment module :
  8300 -   Update status field.
  8301 -   Set the 'published' entity key.

ds module :
  8002 -   Fix ds_switch being added to each content entity.
  8003 -   Uninstalls Layout plugin, then enables Layout Discovery.
  8004 -   Makes sure Layout Discovery is enabled on update, as 8003 could have failed to do so.

image module :
  8201 -   Flush caches as we changed field formatter metadata.

locale module :
  8300 -   Delete translation status data in state.

node module :
  8300 -   Change {node_access}.fallback from an int to a tinyint as it is a boolean.
  8301 -   Set the 'published' entity key.

serialization module :
  8301 -   @see hal_update_8301()
  8302 -   Add serialization.settings::bc_primitives_as_strings configuration.

block module :
  Disable blocks that are placed into the "disabled" region.

system module :
  Update entity displays to contain the region for each field.
  Force caches using hashes to be cleared (Twig, render cache, etc.).
  Force plugin definitions to be cleared.   @see https:www.drupal.orgnode2802663

user module :
  Enforce order of role permissions.

views module :
  Rebuild caches to ensure schema changes are read in

Question, are you guys also using fieldgroup?

I was not using it.

I am using field_group (https://www.drupal.org/project/field_group), too
Same as #22

I am having the issue as originally described. Applying patch in #4 prior to upgrading did not solve my issue. I am using field group, though not necessarily on the layout in question. I'm also using paragraphs. (Anyone else?)

Attached:

1. entity view mode config export for full content, basic page prior to upgrade
2. entity view mode config export for full content, basic page, after upgrade
3. a diff of those two in that order (i.e. it seems the upgrade adds 'region: content' lines to the yml config

I could provide you a composer.json before and after, or anything else you want to see, even complete drush arb backups if you need (privately).

Before upgrade (basic page, full content):

After upgrade (basic page, full content):

Module changes for upgrade (note that since I was switching from drupal-packagist repos to official repos, I 'pinned' all my module versions, only upgrading as needed), so the only modules that changed:

1. core went from 8.2.3 -> 8.3.1 (the fields started shuffling around/disappearing when I when from 8.2.8 to 8.3.1)
2. address went from 1.0-beta3 to 1.0-rc4
3. display suite went from 2.5 to 3.0-beta3
4. before the upgrade, layout_plugin was installed and was alpha22. after the upgrade, it was uninstalled.
5. layout discovery is enabled on the upgraded site.

I don't think I have any other layout modules going on. Mostly using paragraphs. Paragraphs and entity reference revisions are both at 1.0.0.

Let me know how else I can help.

Thnx cwells, that's what I need, I'm using the same setup and don't have any issues so I need to dig deeper....

#25
Looking at your configuration some things stand out:

Before (8.x-2.x):
third_party_settings:ds:regions

regions:
      left:
        - body
      right:
        - field_image
      footer:
        - field_sidebar
        - field_content
        - field_imported_body
        - links

After (8.x-3.x):
third_party_settings:ds:regions

regions:
      left:
        - body
      right:
        - field_image
      footer:
        - field_sidebar
        - field_content
        - field_imported_body
        - links

1. These settings are the same and still valid (like before the update).

Before (8.x-2.x):
content:body:

weight: 0
    label: hidden
    settings: {  }
    third_party_settings: {  }
    type: text_default

content:field_sidebar:

 type: entity_reference_entity_view
    weight: 2
    label: hidden
    settings:
      view_mode: default
      link: false
    third_party_settings: {  }

The field settings do not contain any reference to the region it should appear (which is expected, because of 8.x-2.x).

After (8.x-3.x):
content:body:region:

weight: 0
    label: hidden
    settings: {  }
    third_party_settings: {  }
    type: text_default
    region: content

content:field_sidebar:region:

type: entity_reference_entity_view
    weight: 2
    label: hidden
    settings:
      view_mode: default
      link: false
    third_party_settings: {  }
    region: content

2. The region is wrong for all fields, it should be the list of DS-settings, not "content".
3. The "content" region does not exists. In this example there is: left, right, footer.

Some thoughts/questions:
4. The "good" config is still there in third_party_settings:ds:regions. So this might be fixed with an update hook.
5. Why is third_party_settings:ds:regions still there? Isn't that redundant?

Can someone test this patch?

This appears to be working for me. I re-ran my update to include 8005 and my newest visit to manage display for basic page reads as it had before the upgrade.

Thnx cwells! I'll roll a new release tomorrow.

I got a Invalid argument supplied for foreach() ds.install:82 warning, uploading fix.

@efpapado, I assume everything is working again on your website(s). I pushed this patch and made a new release.
Thnx cwells for the config files.

Edit: Correct image src

Not sure if it is related to this issue or not. Your patch works but I have another issue. After upgrading I get a broken output: broken html code gets printed. I'm attaching screenshots from a teaser, before and after the upgrade.

The screenshots are from the teaser view mode, but it happens also on the default view mode which is used on the node page. I'm not sure what to look for. It seems that the wrapper tag is missing.
I'm not very familiar with twig templating, I did some search on the module's code for the ds_content_wrapper that is printed in ds-1col.html.twig but I couldn't find any php file with that string, in order to try to debug.

Maybe if you could give me a hint, I could try to dig more.

Can you try to resave your layouts in the UI?

It's usually a problem with stale caches or/and preprocess function that aren't called but the last one should be fixed.

I've updated my install with the dev version after the drupal 8.3.1 update and all my display options come back.
I've also clear all caches.

Great job ! Keep on :)

The patch in #4 that I posted still fixes an outstanding issue, opened a new issue #2879326: Custom layouts with a content region display incorrectly in the UI in case anyone is still having similar issues.

I found this issue after upgrading Core to 8.3.x from 8.2.x

we found that our fields were missing entirely from the Manage Display tab (in which we are using DS).

Installing 8.3.x version of DS fixed this issue. HOWEVER, we are still missing the "region" column on the Manage Display screen -- this is the column in which you can select the region to assign your field to with a drop down input.

Is this column supposed to be missing now? Or do i need to make a change somehow to get it to reappear?

Thanks! (Display Suite is by far my favorite module, really love what you all do)

I am having the same issues that everyone else is (Where display suite layouts used to be I am now getting this after updating Drupal Core to 8.3.2) and I believe I have followed all the steps as laid out above. Can someone please help?

< data-history-node-id="45" data-quickedit-entity-id="node/45" class="contextual-region clearfix"> Open configuration options
< data-history-node-id="46" data-quickedit-entity-id="node/46" class="contextual-region clearfix"> Open configuration options
< data-history-node-id="47" data-quickedit-entity-id="node/47" class="contextual-region clearfix">

all my layouts have gone away

Following the above directions I then updated Display Suite to 8.x-3.0, cleared cache and it completely broke my site

Fatal error: Call to a member function getRegions() on array in /code/modules/ds/modules/ds_extras/ds_extras.module on line 23 Call Stack: 0.0035 253368 1. {main}() /code/index.php:0 0.0060 933296 2. Drupal\Core\DrupalKernel->handle() /code/index.php:19 0.0141 2036344 3. Stack\StackedHttpKernel->handle() /code/core/lib/Drupal/Core/DrupalKernel.php:656 0.0141 2036456 4. Drupal\Core\StackMiddleware\NegotiationMiddleware->handle() /code/vendor/stack/builder/src/Stack/StackedHttpKernel.php:23 0.0142 2036848 5. Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle() /code/core/lib/Drupal/Core/StackMiddleware/NegotiationMiddleware.php:50 0.0142 2037280 6. Drupal\page_cache\StackMiddleware\PageCache->handle() /code/core/lib/Drupal/Core/StackMiddleware/ReverseProxyMiddleware.php:47 0.0148 2047632 7. Drupal\page_cache\StackMiddleware\PageCache->pass() /code/core/modules/page_cache/src/StackMiddleware/PageCache.php:78 0.0148 2047712 8. Drupal\Core\StackMiddleware\KernelPreHandle->handle() /code/core/modules/page_cache/src/StackMiddleware/PageCache.php:99 0.0194 2325104 9. Drupal\Core\StackMiddleware\Session->handle() /code/core/lib/Drupal/Core/StackMiddleware/KernelPreHandle.php:47 0.0220 2483632 10. Symfony\Component\HttpKernel\HttpKernel->handle() /code/core/lib/Drupal/Core/StackMiddleware/Session.php:57 0.0221 2484488 11. Symfony\Component\HttpKernel\HttpKernel->handleRaw() /code/vendor/symfony/http-kernel/HttpKernel.php:64 0.0736 4980952 12. Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher->dispatch() /code/vendor/symfony/http-kernel/HttpKernel.php:149 0.0742 5014576 13. Drupal\Core\EventSubscriber\MainContentViewSubscriber->onViewRenderArray() /code/core/lib/Drupal/Component/EventDispatcher/ContainerAwareEventDispatcher.php:111 0.0746 5050064 14. Drupal\Core\Render\MainContent\HtmlRenderer->renderResponse() /code/core/lib/Drupal/Core/EventSubscriber/MainContentViewSubscriber.php:90 0.0746 5050480 15. Drupal\Core\Render\MainContent\HtmlRenderer->prepare() /code/core/lib/Drupal/Core/Render/MainContent/HtmlRenderer.php:117 0.0749 5069736 16. Drupal\Core\Render\Renderer->executeInRenderContext() /code/core/lib/Drupal/Core/Render/MainContent/HtmlRenderer.php:227 0.0749 5069792 17. Drupal\Core\Render\MainContent\HtmlRenderer->Drupal\Core\Render\MainContent\{closure}() /code/core/lib/Drupal/Core/Render/Renderer.php:574 0.0749 5069904 18. Drupal\Core\Render\Renderer->render() /code/core/lib/Drupal/Core/Render/MainContent/HtmlRenderer.php:226 0.0749 5070808 19. Drupal\Core\Render\Renderer->doRender() /code/core/lib/Drupal/Core/Render/Renderer.php:195 0.0751 5091616 20. Drupal\Core\Theme\ThemeManager->render() /code/core/lib/Drupal/Core/Render/Renderer.php:435 0.2000 6448256 21. Drupal\Core\Theme\Registry->getRuntime() /code/core/lib/Drupal/Core/Theme/ThemeManager.php:142 0.2001 6460264 22. Drupal\Core\Utility\ThemeRegistry->__construct() /code/core/lib/Drupal/Core/Theme/Registry.php:252 0.2009 6460672 23. Drupal\Core\Utility\ThemeRegistry->initializeRegistry() /code/core/lib/Drupal/Core/Utility/ThemeRegistry.php:67 0.2009 6460944 24. Drupal\Core\Theme\Registry->get() /code/core/lib/Drupal/Core/Utility/ThemeRegistry.php:86 0.2015 6461728 25. Drupal\Core\Theme\Registry->build() /code/core/lib/Drupal/Core/Theme/Registry.php:232 0.2199 6610320 26. Drupal\Core\Theme\Registry->processExtension() /code/core/lib/Drupal/Core/Theme/Registry.php:333 0.2199 6624336 27. layout_plugin_theme() /code/core/lib/Drupal/Core/Theme/Registry.php:446 0.2213 6712736 28. Drupal\layout_plugin\Plugin\Layout\LayoutPluginManager->getThemeImplementations() /code/modules/layout_plugin/layout_plugin.module:32 0.2213 6712800 29. Drupal\Core\Plugin\DefaultPluginManager->getDefinitions() /code/modules/layout_plugin/src/Plugin/Layout/LayoutPluginManager.php:158 0.2219 6713072 30. Drupal\Core\Plugin\DefaultPluginManager->findDefinitions() /code/core/lib/Drupal/Core/Plugin/DefaultPluginManager.php:174 0.3026 6971256 31. Drupal\Core\Plugin\DefaultPluginManager->alterDefinitions() /code/core/lib/Drupal/Core/Plugin/DefaultPluginManager.php:287 0.3026 6971256 32. Drupal\Core\Extension\ModuleHandler->alter() /code/core/lib/Drupal/Core/Plugin/DefaultPluginManager.php:332 0.3030 6973008 33. ds_extras_layout_alter() /code/core/lib/Drupal/Core/Extension/ModuleHandler.php:501

so through Drush I tried enabling the Layout Discovery module to which I got Layout Discovery cannot be installed because the Layout Plugin module is installed and incompatible.

When I updated the Display Suite module it did not remove the layout plugin module as CWells suggested in #25

What am I missing? Help I am pulling my hair out

Uninstall layout_plugin.
Install layout_discovery.
Clear caches.

If you ran DB updates it should have been fine.