Add a Normalizer and Denormalizer to support Layout Builder

See #2933777: REST/Serializer improvements in core/contrib make it less suitable for default_content use case, layout builder will require special integration as it stores objects, which I personally consider to be a bad idea. if it would store the values in an array format it would just work and also wouldn't require special integration in core rest/jsonapi either, at least not for the basics.

Objects shouldn't be exported, you're going down the wrong path. The correct fix is to normalize it as an array, just like the referenced core issue.

1. +++ b/src/Commands/DefaultContentCommands.php
   @@ -4,13 +4,15 @@
     */
   -class DefaultContentCommands extends DrushCommands {
   +class DefaultContentCommands extends DrushCommands implements ContainerInjectionInterface {
    
      /**
       * The default content exporter.
   @@ -30,6 +32,15 @@
   
   @@ -30,6 +32,15 @@
      }
    
      /**
   +   * {@inheritdoc}
   +   */
   +  public static function create(ContainerInterface $container) {
   +    return new static(
   +      $container->get('default_content.exporter')
   +    );
   +  }
   +
   

   all these create changes are not necessary. they are services and get their dependencies injected automatically.

2. +++ b/src/Normalizer/ContentEntityNormalizer.php
   @@ -253,6 +269,10 @@
            }
   +        elseif ($property instanceof SectionData) {
   +          $section_value = Section::fromArray($value);
   +          $property->setValue($section_value);
   +        }
            else {
              $property->setValue($value);
            }
   @@ -419,6 +439,9 @@
   
   @@ -419,6 +439,9 @@
            $value = $target->uuid();
          }
        }
   +    elseif ($property instanceof SectionData && $property->getValue() instanceof Section) {
   +      $value = $property->getValue()->toArray();
   +    }
   

   yeah, this is the relevant part, that's the only thing that we need here.

   Tests would be great though. Have a look at some of the normalizer tests that I added, if we would do something like that for layout builder, have a test that enables it, and then normalize and denormalize an entity that will make it much easier to get committed.

Based on #22 I created a patch to fix reference on block_content when inserting custom blocks on layout builder.

There is no reference to the block_content uuid, so that patch creates a custom reference to it and make sure the id is correct when importing. It doesn't export the block with "drush dcer" because the block is not really referenced, so you need to manually get the block_content ids from the exported nodes and export them too.

https://www.drupal.org/files/issues/2020-07-22/3160146-18.patch
Works fine for our situation.

#25 is working well in my case

Hello, i've been working with #25 it works on D8 but when i try with D9 the import breaks while importing block_contents with a [InvalidArgumentException]
Property format is unknown.

Any idea if this is a D9 related issue or something in the module?

I adjust patch from #25 and adjust Exporter.php to respect dependencies of the layout builder components so those dependencies gets exported too when using drush default-content:export-references.

Don't now if this is the correct way of doing this and this ticket is the right place for it. Maybe it helps someone.

Hey @nimoatwoodway,

I have just applied #33 and got following errors when importing just regular block_content without layout builder:

web_1            |  [error]  Drupal\Core\Database\IntegrityConstraintViolationException: SQLSTATE[23000]: Integrity constraint violation: 1062 Dupl
icate entry '9f41892c-e577-4514-82d4-c388c21e715a' for key 'block_content_field__uuid__value': INSERT INTO "block_content" ("revision_id", "type", 
"uuid", "langcode") VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3); Array         
web_1            | (                                                                                                                               
web_1            |     [:db_insert_placeholder_0] =>                                                                                               
web_1            |     [:db_insert_placeholder_1] => limited_html_block                                                                            
web_1            |     [:db_insert_placeholder_2] => 9f41892c-e577-4514-82d4-c388c21e715a                                                          
web_1            |     [:db_insert_placeholder_3] => en                                                                                            
web_1            | )                                                                                                                               
web_1            |  in Drupal\Core\Database\Driver\mysql\ExceptionHandler->handleExecutionException() (line 47 of /var/www/html/web/core/lib/Drupal
/Core/Database/Driver/mysql/ExceptionHandler.php).                                                                                                 
web_1            |                                                                                                                                 
web_1            | In SqlContentEntityStorage.php line 810:                                                                                        
web_1            |                                                                                                                                 
web_1            |   SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '9f41                                                   
web_1            |   892c-e577-4514-82d4-c388c21e715a' for key 'block_content_field__uuid__value                                                   
web_1            |   ': INSERT INTO "block_content" ("revision_id", "type", "uuid", "langcode")                                                    
web_1            |   VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_plac                                                   
web_1            |   eholder_2, :db_insert_placeholder_3); Array                                                                                   
web_1            |   (                                                                             
web_1            |       [:db_insert_placeholder_0] =>                                                                                             
web_1            |       [:db_insert_placeholder_1] => limited_html_block                                                                          
web_1            |       [:db_insert_placeholder_2] => 9f41892c-e577-4514-82d4-c388c21e715a       
web_1            |       [:db_insert_placeholder_3] => en                                         
web_1            |   )                                                                            
web_1            |                                                                                 
web_1            |                                                                                                                                 
web_1            | In ExceptionHandler.php line 47:                                                                                                
web_1            |                                                                                 
web_1            |   SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '9f41  
web_1            |   892c-e577-4514-82d4-c388c21e715a' for key 'block_content_field__uuid__value  
web_1            |   ': INSERT INTO "block_content" ("revision_id", "type", "uuid", "langcode")   
web_1            |   VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_plac  
web_1            |   eholder_2, :db_insert_placeholder_3); Array                                   
web_1            |   (                                                                             
web_1            |       [:db_insert_placeholder_0] =>                                             
web_1            |       [:db_insert_placeholder_1] => limited_html_block                          
web_1            |       [:db_insert_placeholder_2] => 9f41892c-e577-4514-82d4-c388c21e715a       
web_1            |       [:db_insert_placeholder_3] => en                                          
web_1            |   )                                                                             
web_1            |                                                                                 
web_1            | 
web_1            | In StatementWrapper.php line 116:
web_1            |                                                                                 
web_1            |   SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '9f41  
web_1            |   892c-e577-4514-82d4-c388c21e715a' for key 'block_content_field__uuid__value  
web_1            |   '                                                                             
web_1            |                                                                                 
web_1            | 
web_1            | run-parts: /usr/lib/fruition/bootstrap/400-scaffold.sh exited with return code 1

Hey @nimoatwoodway,

Never-mind my previous comment, looks like I messed up my local install previously, I just fixed it and re-tested #33, it worked without problems, thank you!

+++ b/src/Normalizer/ContentEntityNormalizer.php
@@ -253,6 +255,22 @@ class ContentEntityNormalizer implements ContentEntityNormalizerInterface {
+            if (isset($component['configuration']['id'])) {
+              [$component_type, $component_bundle] = explode(PluginBase::DERIVATIVE_SEPARATOR, $component['configuration']['id']);

This assumes we always have a derived plugin. But custom blocks that are exposed to LB don't always have this. NW for that.

Fixed #36.

The other section in the patch still has the same assumption that each block has a derivative ID, why not update that too?

@heddn, your interdiff is incorrect.
Your patch remove the dependency management in Exporter.php file.

With your patch in #37, the "dcer" drush command no longer exports the inline blocks contributed in content.

Patch #33 modify Exporter.php to manage dependencies. Without the lines below, my install throw errors of unexisting block_content.

diff --git a/src/Exporter.php b/src/Exporter.php
index e0df909..2aafb36 100644
--- a/src/Exporter.php
+++ b/src/Exporter.php
@@ -197,8 +197,31 @@ class Exporter implements ExporterInterface {
    *   Keyed array of entities indexed by entity type and ID.
    */
   protected function getEntityReferencesRecursive(ContentEntityInterface $entity, $depth = 0, array &$indexed_dependencies = []) {
+
     $entity_dependencies = $entity->referencedEntities();
 
+    // Get dependencies from layout builder.
+    if($entity->hasField('layout_builder__layout')) {
+      $section_data = $entity->get('layout_builder__layout');
+      foreach ($section_data->getSections() as $section) {
+        $components = $section->getComponents();
+        foreach ($components as $component) {
+          $plugin = $component->getPlugin();
+          if ($plugin instanceof \Drupal\layout_builder\Plugin\Block\InlineBlock) {
+            $configuration = $component->get('configuration');
+            if(!empty($block_revision_id = $configuration['block_revision_id'])) {
+              $block_content = $this->entityTypeManager
+                ->getStorage('block_content')
+                ->loadRevision($block_revision_id);
+              if($block_content) {
+                $entity_dependencies[] = $block_content;
+              }
+            }
+          }
+        }
+      }
+    }
+

In your update of default contents in Verbase profile, you haven't "depends" property in your node content YAML file.
You cann't be sure, your content entities used in your node YAML file already exist.

Patch #35, add this lines in my local node YAML file (non exhaustive list) :

  depends:
    eebda9cd-2759-45a8-90f2-41db86649a92: block_content
    f535b39f-d0a8-42e3-831f-cb487fdca8dd: block_content
    0b8ebf54-ddbc-4d98-b564-8d98fbc30c71: block_content
    a4abfb29-ece7-4995-b242-9d0b7ee92572: block_content

My blocks content (inline and reusable) are now created before to create node.

+1 for RTBC

Tested Patch #33 in conjunction with the following patches (FWIW both of these patches were applied before patch #33):

• Issue #2698425: Update existing entities instead of trying to save them as new ones
• Issue #3176839: Don't add translation, if exist

I had no problem importing and exporting both inline and regular custom block content.

Confirming @lhridley series of patches.

Applied patches on both a Drupal 8 and Drupal 9 build.

Export from Drupal 8
Import into Drupal 9

Was successful!

Hope that all three patches are applied to a release someday!

Just installed patch #37 to avoid warnings described in #36

No problems so far. Thanks!

Tried applying patch #37.Also tried applying multiple patches as mentioned in the comment #42 .Both cases are not working.I'm still getting this error while exporting contents.

Created a MR with #37 and the entity reference fix from #41. Export and reinstall works perfectly.

Installed and tested patch from MR
Displaying /layout page of a node using LB content override and exported/imported using DC module throws an error:

http://172.31.0.4/node/1/layout
Referrer	http://172.31.0.4/lb
Message	Error: Call to a member function getEntityTypeId() on null in Drupal\layout_builder\Plugin\Block\InlineBlock->build() (line 218 of /var/www/html/web/core/modules/layout_builder/src/Plugin/Block/InlineBlock.php)
#0 /var/www/html/web/core/modules/layout_builder/src/EventSubscriber/BlockComponentRenderArray.php(106): Drupal\layout_builder\Plugin\Block\InlineBlock->build()
#1 [internal function]: Drupal\layout_builder\EventSubscriber\BlockComponentRenderArray->onBuildRender()
#2 /var/www/html/web/core/lib/Drupal/Component/EventDispatcher/ContainerAwareEventDispatcher.php(142): call_user_func()
#3 /var/www/html/web/core/modules/layout_builder/src/SectionComponent.php(90): Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher->dispatch()
#4 /var/www/html/web/core/modules/layout_builder/src/Section.php(86): Drupal\layout_builder\SectionComponent->toRenderArray()
#5 /var/www/html/web/core/modules/layout_builder/src/Element/LayoutBuilder.php(240): Drupal\layout_builder\Section->toRenderArray()
#6 /var/www/html/web/core/modules/layout_builder/src/Element/LayoutBuilder.php(124): Drupal\layout_builder\Element\LayoutBuilder->buildAdministrativeSection()
#7 /var/www/html/web/core/modules/layout_builder/src/Element/LayoutBuilder.php(98): Drupal\layout_builder\Element\LayoutBuilder->layout()
#8 [internal function]: Drupal\layout_builder\Element\LayoutBuilder->preRender()
#9 /var/www/html/web/core/lib/Drupal/Core/Security/DoTrustedCallbackTrait.php(101): call_user_func_array()
#10 /var/www/html/web/core/lib/Drupal/Core/Render/Renderer.php(772): Drupal\Core\Render\Renderer->doTrustedCallback()
#11 /var/www/html/web/core/lib/Drupal/Core/Render/Renderer.php(363): Drupal\Core\Render\Renderer->doCallback()
#12 /var/www/html/web/core/lib/Drupal/Core/Render/Renderer.php(435): Drupal\Core\Render\Renderer->doRender()
#13 /var/www/html/web/core/lib/Drupal/Core/Render/Renderer.php(201): Drupal\Core\Render\Renderer->doRender()
#14 /var/www/html/web/core/lib/Drupal/Core/Render/MainContent/HtmlRenderer.php(241): Drupal\Core\Render\Renderer->render()
#15 /var/www/html/web/core/lib/Drupal/Core/Render/Renderer.php(564): Drupal\Core\Render\MainContent\HtmlRenderer->Drupal\Core\Render\MainContent\{closure}()
#16 /var/www/html/web/core/lib/Drupal/Core/Render/MainContent/HtmlRenderer.php(242): Drupal\Core\Render\Renderer->executeInRenderContext()
#17 /var/www/html/web/core/lib/Drupal/Core/Render/MainContent/HtmlRenderer.php(132): Drupal\Core\Render\MainContent\HtmlRenderer->prepare()
#18 /var/www/html/web/core/lib/Drupal/Core/EventSubscriber/MainContentViewSubscriber.php(90): Drupal\Core\Render\MainContent\HtmlRenderer->renderResponse()
#19 [internal function]: Drupal\Core\EventSubscriber\MainContentViewSubscriber->onViewRenderArray()
#20 /var/www/html/web/core/lib/Drupal/Component/EventDispatcher/ContainerAwareEventDispatcher.php(142): call_user_func()
#21 /var/www/html/vendor/symfony/http-kernel/HttpKernel.php(163): Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher->dispatch()
#22 /var/www/html/vendor/symfony/http-kernel/HttpKernel.php(80): Symfony\Component\HttpKernel\HttpKernel->handleRaw()
#23 /var/www/html/web/core/lib/Drupal/Core/StackMiddleware/Session.php(58): Symfony\Component\HttpKernel\HttpKernel->handle()
#24 /var/www/html/web/core/lib/Drupal/Core/StackMiddleware/KernelPreHandle.php(48): Drupal\Core\StackMiddleware\Session->handle()
#25 /var/www/html/web/core/modules/page_cache/src/StackMiddleware/PageCache.php(106): Drupal\Core\StackMiddleware\KernelPreHandle->handle()
#26 /var/www/html/web/core/modules/page_cache/src/StackMiddleware/PageCache.php(85): Drupal\page_cache\StackMiddleware\PageCache->pass()
#27 /var/www/html/web/core/lib/Drupal/Core/StackMiddleware/ReverseProxyMiddleware.php(48): Drupal\page_cache\StackMiddleware\PageCache->handle()
#28 /var/www/html/web/core/lib/Drupal/Core/StackMiddleware/NegotiationMiddleware.php(51): Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle()
#29 /var/www/html/web/modules/contrib/remove_http_headers/src/StackMiddleware/RemoveHttpHeadersMiddleware.php(49): Drupal\Core\StackMiddleware\NegotiationMiddleware->handle()
#30 /var/www/html/vendor/stack/builder/src/Stack/StackedHttpKernel.php(23): Drupal\remove_http_headers\StackMiddleware\RemoveHttpHeadersMiddleware->handle()
#31 /var/www/html/web/core/lib/Drupal/Core/DrupalKernel.php(708): Stack\StackedHttpKernel->handle()
#32 /var/www/html/web/index.php(19): Drupal\Core\DrupalKernel->handle()
#33 {main}
.

Note that a Inline block content was added to layout

Hi,

Uploading patch from MR 15 to be able to add it in Composer.

Hello,

I rebased the MR and made a small fix. When parsing layout builder blocks and a non-derivative plugin is found, an unused variable is not defined provoking PHP warnings.

Here is the patch for easier composer usage.

Does this patch also fix, Layout Paragraphs Support?

@SocialNicheGuru in reply to #57 I tested this and no I don't think it fixes Layout Paragraphs Support

+1 for the RTBC.

Can we get a basic test for that?

Hi,

For information: #3359137: Update inline block usage on import

Current approach just avoids exception & skips "block serialized" approach, which is the default one; @see "\Drupal\layout_builder\Plugin\Block\InlineBlock::getEntity()":

...
if (!empty($this->configuration['block_serialized'])) {
  $this->blockContent = unserialize($this->configuration['block_serialized']);
}
elseif (!empty($this->configuration['block_revision_id'])) {
  $entity = $this->entityTypeManager->getStorage('block_content')->loadRevision($this->configuration['block_revision_id']);
  $this->blockContent = $entity;
}
....

So any serialized block dependencies are ignored, and is the first option when new non existing content block is used in layout builder, besides that approach just works with "inline_block" plugin variants & "InlineBlock instance" can be used instead in order to get it working per base class.

Other case where that should be managed as in layout builder you can mix entity field blocks and content blocks (content) into any display of exported content or its dependencies.

Also some entities like section library template may use same field type to store sections but in different field name, not "layout_builder__layout" but "layout_section", to manage that again "instance of SectionData" can be used instead to detect it.

So taking all use cases would be something like:

1. Content blocks serialized & dependent entities (media or other)
2. Content blocks direct usage & dependent entities (not serialized)
3. Content blocks usage inside any display of exported entity or its dependent entities. Maybe this one is the hardest to manage. Not display config entity but any content block usage on it.

Besides changes going in this direction should be compatible with sites without layout_builder module enabled? Maybe all should be managed into import/export subscribers.

Working into patch/mr to manage at least first & second use cases.

To clarify main point, is not just about normalize / denormalize field but also include dependencies (exportContentWithReferences).

Added changes to MR;

1. Avoid layout_builder classes direct use to not force module dependency by missing classes on sites without layout builder enabled
2. Manage serialized block content
3. Use base classes to evaluate instances to avoid strict unique core lb support

Maybe "Drupal\block_content\Plugin\Block\Drupal\block_content\Plugin\Block" should be also supported (direct block content usage from core content block library). Also other possible use cases like commented into #63 -> 3.

Thanks for patch #53, we have tested it in drupal 9.5. And it works correctly.

Would #2703607: drush command default-content-export-module-with-references help too?

The content looks like it exports correctly but then I get this error on import:

In SectionData.php line 35:

  [InvalidArgumentException]
  Value assigned to "section" is not a valid section

I think the function is here in core/modules/layout_builder/src/Plugin/DataType/SectionData.php:

public function setValue($value, $notify = TRUE) {
    if ($value && !$value instanceof Section) {
      throw new \InvalidArgumentException(sprintf('Value assigned to "%s" is not a valid section', $this->getName()));
    }
    parent::setValue($value, $notify);
  }

$value is coming in as an array not instanceof Section.

Added test for new normalizer, let me know if I can help more, thanks

Latest patch is working nicely on Drupal 10.3.6.. Both import and export of a Layout Builder page work as expected.

Not sure if this is the correct place to report this. I'm also getting the Object support when dumping a YAML file has been disabled. error. I saw that another post was closed that reported this error as being a duplicate. We're using layout-paragraphs (gasp) but that hasn't been the issue as we have been successful in the past in exporting and importing this data. We are actually still using the import functionality from our last export nearly a year or two ago but need to update our default content.

I installed the most recent patch and the error occurs at the $return = Yaml::encode($normalized) in the Exporter.php class. Since we have a lot of stuff being exported out I added the following to help me identify the issue

    function find_objects($array, $path = '') {
      foreach ($array as $key => $value) {
        $new_path = $path ? $path . '.' . $key : $key;
        if (is_object($value)) {
          echo "Found object at $new_path\n";
        } elseif (is_array($value)) {
          find_objects($value, $new_path);
        }
      }
    }

    find_objects($normalized);

And it looks like this is erroring out for us on a meta tag breadcrumb item we have on our page. I've included a screenshot from the xdebug if this might be helpful.

Any recommendations?

**Edit:
It's definitely the meta tag open graph webpage breadcrumb settings. I turned off the setting and I can export without that error. I have a workaround now but wanted to report this issue. Thank you @rajab_natshah for your work on the patch . . I would never have found this issue without seeing the files you touched.

This is going to be important for some recipes in Drupal CMS and was referenced in #3493287: The default content importer should handle Layout Builder section data

Thanks for the reference. I would expect that this simplifies this issue a lot then, might be enough to normalize/export the structure into the correct format, without special import handling.

I'm having some issues with exporting Layout Builder layouts containing blocks of type 'inline_block'.
The patch in this issue allows them to be exported, and then imported via a Recipe (Drupal core behaviour).

However, in the export there is an explicit numeric 'block_id' and 'block_revision_id' property.
On the website I'm importing the content via a Recipe, the inline blocks don't necessarily receive the exact same numeric IDs.

Ideally the default content should reference UUIDs, instead of numeric entity IDs.

From my first tests, I'm able get this working with the patch in #3180702: Export block UUID with custom Layout Builder blocks (both the exporting & importing site need this patch applied).

Not sure why latest changes was reverted, recover most relevant notes about important points written in #63 & #65,
please note that any existing exported default content generated without this logic, or previous version, may be incorrect as it is expectable and may be regenerated:

Current approach just avoids exception & skips "block serialized" approach, which is the default one; @see "\Drupal\layout_builder\Plugin\Block\InlineBlock::getEntity()":

...
if (!empty($this->configuration['block_serialized'])) {
  $this->blockContent = unserialize($this->configuration['block_serialized']);
}
elseif (!empty($this->configuration['block_revision_id'])) {
  $entity = $this->entityTypeManager->getStorage('block_content')->loadRevision($this->configuration['block_revision_id']);
  $this->blockContent = $entity;
}
....

So any serialized block dependencies are ignored, and is the first option when new non existing content block is used in layout builder, besides that approach just works with "inline_block" plugin variants & "InlineBlock instance" can be used instead in order to get it working per base class.

Other case where that should be managed as in layout builder you can mix entity field blocks and content blocks (content) into any display of exported content or its dependencies.

Also some entities like section library template may use same field type to store sections but in different field name, not "layout_builder__layout" but "layout_section", to manage that again "instance of SectionData" can be used instead to detect it.

So taking all use cases would be something like:

1. Content blocks serialized & dependent entities (media or other)
2. Content blocks direct usage & dependent entities (not serialized)
3. Content blocks usage inside any display of exported entity or its dependent entities. Maybe this one is the hardest to manage. Not display config entity but any content block usage on it.

Reverted changes to MR;

1. Avoid layout_builder classes direct use to not force module dependency by missing classes on sites without layout builder enabled
2. Manage serialized block content
3. Use base classes to evaluate instances to avoid strict unique core lb support

Maybe "Drupal\block_content\Plugin\Block\Drupal\block_content\Plugin\Block" should be also supported (direct block content usage from core content block library). Also other possible use cases like commented into #63 -> 3.

Happily using those changes for months with Layout builder, VLSuite & Section library...

Attach patch of that mentioned changes in case anyone wants to try, please also note that content provided by recipe & generated by default content will not use the same logic so it will fail so... maybe something has also to be done into "core recipe import content logic" to use similar or compatible strategies?

It should work fine using genenerated default content using these changes & creating content also with these changes (default content not core recipe content).

It is worth mentioning that exporting layout data is natively supported by the default content exporter that is now in core and will be released in 11.3.0. (See https://www.drupal.org/node/3533854 for info about that.)

Noting that content export from core fails to export pathauto configuration property for the node.

Thanks @crzdev for the patch. In our case, we use both layout builder (works partially in core, requires patch in default_content) as well as paragraphs (supported by default_content, but not by core #3553119: Drupal content export does not support nested entities - Layout Builder inline blocks, paragraphs). Your patch saved us ;)

This duality is a bit troublesome, are there any plans to combine these efforts?

The plan is pretty simple: deprecate this module in favor of core after ensuring feature parity with the integrations and modules this supports. There are a few open issues, including one in ERR and one here to run our tests against the core API.

No plans on when that will happen, right now it would essentially be up to me (or someone else) to push it forward (in my case in my free time), if someone is willing to sponsor some time for that, reach out to me.