Problem/Motivation
When accessing Layout Builder overrides via REST (i.e. when accessing an overridden entity's layout field), the contents of sections are empty.
Proposed resolution
Add a normalizer for the layout_section DataType, which ensures that sections can be properly serialized and unserialized via the serialization API (and REST).
Remaining tasks
N/A
User interface changes
N/A
API changes
A new normalizer will be added for the layout_section DataType.
Layout section data will be exposed for all serialization types.
Data model changes
N/A
| Comment | File | Size | Author |
|---|
Issue fork drupal-2942975
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #2
tim.plunkettComment #3
samuel.mortensonHere's a start to this, still need to add tests and think about validation.
I opted to write two normalizers - one for the layout_section DataType and another for the field item. This made sense to me as it allows for new kinds of layout field, but we could put everything in the field item normalizer if that seem simpler. I tested GET/PATCH and it seems to work, so feel free to test things out and find bugs as I (or others) write test coverage.
Comment #4
wim leersGreat start :)
@DataType-level normalizer! This is in line with #2926507: Discourage @FieldType-level normalizers, encourage @DataType-level normalizers, to strengthen the API-First ecosystem! It also means this will work with JSON API :)Why these priorities? We've been insisting that normalizers always have a comment explaining the rationale behind their priorities.
(This is a huge weakness in the Symfony DIC — priorities should not be numerical; one should have to define before/after dependencies. Numerical priorities don't work in projects that can be extended by simply installing modules.)
Ideally we wouldn't have
@FieldType-level normalizers.But it's indeed necessary for denormalization.
Can you please make this service private and internal? So we can remove it when #2957385: FieldItemNormalizer never calls @DataType-level normalizer service' ::denormalize() method lands.
I'm a bit scared by how freeform this seems to be. If it's so freeform, how will https://www.drupal.org/project/openapi then be able to generate documentation for this?
Comment #5
samuel.mortensonAddressed all points in #4 (including new unit test coverage 🏄♂️) except the validation and schema feedback. May need @tim.plunkett to chime in here for direction.
Comment #7
samuel.mortensonFixed coding standards on the new test.
Comment #8
samuel.mortensonAnd updated a test method to actually test ::supportsDenormalization
Comment #10
samuel.mortensonMissed one.
Comment #13
tim.plunkettnamespace mismatch; delete the
\Normalizerpart or move the file in and things should pass.Comment #14
samuel.mortensonFixed the namespace issue, thanks @tim.plunkett!
Comment #15
wim leersGreat work!
❤️
This is tying the denormalization of a
@DataTypeto its containing field. That shouldn't be necessary. All this should be doing is creating the value necessary to be set on the main property of a@FieldType=layout_section.Then
LayoutSectionItemNormalizer::denormalize()can take that return value and set it on the@FieldType=layout_sectiontarget instance.Is this confusing, messy, and poorly delineated? Yes. That's what #2926507: Discourage @FieldType-level normalizers, encourage @DataType-level normalizers, to strengthen the API-First ecosystem aims to fix/standardize, but until then we need to be vigilant, to not introduce even more of these.
🤘
Comment #16
borisson_Based on #15, we need to fix .2 of that review.
Comment #17
samuel.mortensonWe discussed this in a private chat already, but I don't like the idea of having a typed data normalizer that normalizes
\Drupal\layout_builder\Plugin\DataType\SectionData, but does not return an instance of SectionData when denormalized. Normalizers are meant to accept their::supportedInterfaceOrClassin normalize, and return an object of that class in denormalize based on the$classargument. Changing this behavior defeats the purpose of normalization for me, even if core is already doing this.Maybe a middle ground would be to not have a typed data normalizer for SectionData, and instead write one for
Drupal\layout_builder\Section, which is the value it accepts. Then the field normalizer could use this to set a value on its typed data object.Comment #18
wim leersI'm not sure I agree with #2914503: Determine if serializing blobs is safe for use in Layout Builder's outcome — if that had had a different outcome, then this issue may not have needed to exist.
Comment #20
mpotter commentedAdding this as a stable blocker as sites need a way to export landing page content for deployment to other environments. This was possible with Lightning Landing Pages with Panelizer and is needed for nodes using Layout Builder with modules such as Default Content. Default Content exports are empty without this serializing patch.
Comment #21
mpotter commentedHere is a reroll of #14 against the latest 8.6
Comment #22
mpotter commentedComment #23
mpotter commentedNot sure if this is working for REST, but it's not working for default_content. This patch allows me to export a node with layout_builder and I see the proper json data in the export. However when I try to import the node json, I get:
When debugging this, it's trying to assign an array to the "section" property rather than a Section object class. Doesn't seem like the denormalizer is being called in LayoutSectionItemNormalizer::denormalize. But maybe it hasn't gotten to that point yet since the debug stack shows it is trying to get the property value in order to determine if it's empty first.
Seems maybe related to this: https://www.drupal.org/project/jsonapi/issues/2955615 in JSONApi, but not fixed in core itself.
Edited: Looks like this is what @samuel.mortenson was saying in #17, so sorry for the dup.
Comment #24
mpotter commentedHere is a tweak to the patch. This doesn't feel right at all, but it got me past the errors in default_content when importing the section json. default_content import is never calling anything in the SectionDataNormalizer so not sure if that's an issue with default_content or with the normalizer. Given that Sam mentioned he tested the previous patch with JSON get/patch it makes me think either default_content has a problem with json importing typed-data or that there is another core issue buried somewhere.
In any case, posting this patch so a client can try it and to get more comments.
Comment #25
wim leers#24: Yeah, it definitely doesn't seem right that
LayoutSectionItemNormalizer::denormalize()is duplicating logic fromSectionDataNormalizer::denormalize(). That almost must mean thatdefault_contentdoes something funky, like only denormalizing field-level data and not field property-level data.Re-scanning this issue, I see that in #18 I linked to #2914503: Determine if serializing blobs is safe for use in Layout Builder. That reminds me of #2895532: @DataType=map cannot be normalized, affects @FieldType=link, @FieldType=map.
Comment #26
huzookaComment #27
jibranLet's add some docs here.
We should add dedicated test(maybe KTB) for this.
Comment #28
gun_dose commentedI applied patch from #24 to Drupal 8.6.4 and it works fine for me. I exported node with layout to default content and imported it back and all works fine.
Comment #29
jidrone commentedThe #24 also worked for me.
Comment #30
wim leersThis just came up in the JSON:API issue queue: #3022575: [upstream] Support for layout 😀
Comment #31
tallytarik commented#24 also worked for me, Drupal 8.6.5.
Comment #32
jurgenhaasTried #24 in 8.6.7 and it works as expected.
Comment #33
wim leersSorry for the long silence on my side 😔 A proper deep dive below!
👍 Normalization:
First, and most importantly: applying this patch makes it show up in the
jsonandhal_jsonformats fore core's REST module, as well as for the contrib JSON:API module (soon to be in core: #2843147: Add JSON:API to core as a stable module). All information is available, and consistently so.🙏 Normalization test coverage
This patch contains unit test coverage (👏), but not functional test coverage. Experience has taught me that for anything REST API-related, we want functional test coverage. Because there are so many subsystems being integrated (Entity API, Field API, Typed Data API, Routing API, Access API, Serialization API …), a functional test is a way to prove that it all actually works in actual requests.
#2942976: Add REST test coverage for Layout Builder already exists for this, but I think it should be merged with this. We already have examples of REST tests for specific field types:
\Drupal\Tests\datetime\Functional\EntityResource\EntityTest\EntityTestDateonlyTest,\Drupal\Tests\entity_test\Functional\Rest\EntityTestTextItemNormalizerTest, and more.🤔 Schema: documentation describing how to interpret the stored value
Second most important: being able to make sense of this data: what is its schema, what is the meaning of the structure, and each particular value?
Without this, it's going to be
impossibledifficult for decoupled applications to allow a content creator to customize each individual entity and have the client respect this custom layout.Now, to be fair, Drupal 8.0.0 shipped without schema support for its REST API. I don't think we should hold Layout Builder to a higher standard. So I'm not arguing for complete/comprehensive API docs. I'm saying that for this field to be actually consumable/interpretable, given the complexity of a value for this field/data type is significantly higher than other core fields, it's necessary for it to be at least clearly documented. https://www.drupal.org/docs/8/core/modules/layout-builder does not yet exist.
#2919811: Write Layout Builder handbook documentation already exists for this.
😨 Validation
It seems that validation is currently limited to:
@mpotter encountered this in #23 when trying to use Layout Builder + https://www.drupal.org/project/default_content. Which is why he added the tweak/hack/work-around in #24.
This isn't "real" validation though: it's only verifying that it receives data of a certain type, it doesn't yet verify the values of that data.
I've searched through the code base, and as far as I can tell, validation is absent! 😨 To be more precise: there is no active need for validation logic, because the UI is structured in such a way that only trusted logic ever generates values to be stored.
\Drupal\layout_builder\SectionListInterface+\Drupal\layout_builder\Section+\Drupal\layout_builder\SectionComponentfit together, forming a data structure that is more restricted than just an array, the UI/form logic interacts with methods on these, and hence there hasn't been a need for validation logic (at least not yet).However, when a decoupled client wishes to modify the layout for an entity, it of course does not have access to these nice PHP interfaces — all it has is plain JSON. There is no validation constraint on either
@FieldType=layout_sectionor@DataType=layout_section. Which means that I can set arbitrary data. This has at least two dangerous consequences:😨 Access
AFAICT currently anybody who can edit the entity, can also edit Layout Builder's field.
Considering the risks wrt validation, a viable strategy could be that only trusted users can modify Layout Builder's field. There is a
configure any layoutpermission, but it's only applied to routes (with the_has_layout_sectionroute requirement).@FieldType=layout_sectioncould be protected by this permission as well. Created #3028301: Do not expose to Layout Builder's sections either in defaults or overrides to REST and other external APIs for this.🙏 Normalization implementation: complicated by the desire to support writes
The only reason that
LayoutSectionItemNormalizerexists (at least until #2957385: FieldItemNormalizer never calls @DataType-level normalizer service' ::denormalize() method lands), and also the only reason that\Drupal\layout_builder\Plugin\DataType\SectionDataneeds to be modified, is to support writes. The patch would become much simpler if we chose to not support writing, for now. Most of the value is in reading it anyway!Conclusion
I see the following paths forward:
inaccessible (even better!) would instantly solve this issue.internalfor now (see https://www.drupal.org/node/2916592)See
2942975-33_forbid_all_access.patch.See
2942975-33_forbid_write_access.patch.See #3028301: Do not expose to Layout Builder's sections either in defaults or overrides to REST and other external APIs.
Comment #34
tedbow@Wim Leers thanks for the detailed review!
I think this is similar to the block UI in that if you give the user administer blocks permission that can place any block. True the user could set a visibility rule for user role but they don't have to. So you are essentially giving them permission to place any block and have render whatever it will render.
At least in the layout case, for now, configure any layout, is a restricted permission.
So for a Layout user they can place any block regardless of whether they are doing so via the UI or through a REST call.
Yes this is true for except for the access control on the routes. With the route access control only users with configure any layout permission can edit layouts. They don't actually need access to update the entity(or view right now 😫#3028490: Users with "configure any layout" can see entities they don't have "view" access to)
I 2nd this. I think a read-only solution now would be very useful.
I would probably defer to @tim.plunkett on this. I guess 1 reason to not expose the field at all right now is I think we would be much more sure after 6 months or 1 year of Layout Builder being stable.
I think that if for some reason we need to change the way the field is stored we could do that without any BC implications if the field is not exposed via our APIs. Because the no module should ever deserializing the Layout reading the data.
If we expose the field over the API it would be effectively just that same as saying we support modules getting the json string decoding it and figuring out the layout from there. Because this is effectively be what a decoupled application would be doing.
And once we expose we can never change that structure. Though in theory we could write a BC layer to with normalizer to keep the format the same for sites that want it the same. But considering how much discussion there was around #2751325: All serialized values are strings, should be integers/booleans when appropriate and #2768651: Let TimestampItem (de)normalize to/from RFC3339 timestamps, not UNIX timestamps, for better DX I think that is something we want to avoid at all costs if we can.
So that would lead me to endorse:
If we can't do that I think we should only allow read access for now.
Comment #35
tedbowI have repurposed the existing issue which was Add default field-level access restriction to Layout Builder's field type to #3028301: Do not expose to Layout Builder's sections either in defaults or overrides to REST and other external APIs
basically the idea is do this in 3 stages
but we could discuss on that issue and leave this 1 for when we actually want to implement the normalizer(if we don't want to do it now)
Comment #36
tim.plunkettMoving tag to other issue
Comment #37
wim leers#34:
If that were the case, I wouldn't be worried. But today, there's no access control on the layout field at all: if the entity can be edited, then the layout can be edited. Which you acknowledged further in your comment.
Thanks for linking #3028490: Users with "configure any layout" can see entities they don't have "view" access to — I hadn't found that yet. Following.
Marking this issue postponed on #3028301: Do not expose to Layout Builder's sections either in defaults or overrides to REST and other external APIs.
Comment #39
xjmComment #40
xjmTurns out we had to introduce a hack in JSON:API itself to avoid exposing LB data over JSON:API. Adding #3040354: Introduce entity type annotation key for indicating that an entity type is *safe* for HTTP API usage as a related issue.
Since our existing normalizer for REST didn't apply to JSON:API, retitling to make it clear that fixing both should be in scope.
Comment #41
andypostThe only issue I recall is inline blocks which has no ui and not enough data exposed
Comment #42
xjm@Wim Leers filed #3043245: Allow marking a config schema types to be marked as "internal" (to avoid exposing them via HTTP APIs).
Comment #43
mrweiner commentedJust curious whether any progress has been made here. Since both JSON:API and Layout Builder are stable in core I think people will expect them to play nicely together.
Comment #44
mpotter commentedAlso wondering what the current workaround for this is. The original patch in #24 was to provide a Json Normalizer for SectionData so the default_content module could export landing page nodes to be deployed across sites.
In the past, landing pages could be designed using Panels and Page Manager and exported to config and deployed across site environments. With Layout Builder, landing pages are Node content and the main way our clients deployed this content was via the default_content module. Now with 8.7 this has all broken and clients no longer have a way to deploy configured landing pages between environments (such as a search results page with facet blocks and other blocks placed on the page).
Is there any other way to restore a Json Normalizer that default_content can use to export a Node with Layout SectionData as JSON without exposing it to the world via Json:api?
Comment #45
andypost@mpotter for a while this normalizers could be added to https://www.drupal.org/project/better_normalizers
Comment #46
larowlanAdded duplicate issue from Default Content to related issues
Comment #47
sharique commentedI'm using 8.7.1, patch #33 doesn't work for me.
And patch #24 give following error.
Error: Maximum function nesting level of '256' reached, aborting! in Drupal::getContainer() (line 128 of /var/www/centara/docroot/core/lib/Drupal.php) #0 /var/www/centara/docroot/core/lib/Drupal.php(283): Drupal::getContainer() #1 ......Both patches work for me neither in JSON API nor default content export.
Comment #48
mrweiner commented@Sharique I haven't actually tested the patches yet myself, but the error you've noted is unrelated. See https://stackoverflow.com/questions/4293775/increasing-nesting-function-...
Comment #49
gun_dose commentedIs there any progress? At 8.7 version all patches are not working.
Comment #50
thursday_bw commentedLike gundose stated.
The patches on this page don't work with 8.7.
I have applied the patch in #24 cleanly, however there is no layout in the exported landing page.
Comment #51
thursday_bw commentedIf i'm understanding all this correctly,
layouts aren't exposed to REST and other API's in drupal 8.7.* due to https://www.drupal.org/project/drupal/issues/3028301 which in turn means, default_content_deploy doesn't export it either. Even with the patches on the issue this comment refers to being applied.
Can anyone thing of a work around, even if temporally?
Comment #52
thursday_bw commentedOK.. Work around.
I'm uploading this patch but I do not recommend that it be accepted into core.
It's just here for you to apply to D8.7 if you so choose as a work around.
If you apply patch 2942975-24.patch
along with 2942975-52.patch
You will be able to export the layout using
default content deploy
Due to https://www.drupal.org/project/drupal/issues/3028301 restricting exposure of layouts, I don't know yet what a correct solution to this
would be.
Comment #53
andypostI think better to add normalizer to https://www.drupal.org/project/better_normalizers and create separate release for 8.7
Comment #54
luke_nuke commentedThis issue blocks layouts from being exportable by any module attempting that (default_content, content_sync among others). As most people from issue queues of these modules will end up here, I decided to write up the possible solution here.
If you really want to have this working on 8.7.x (and possibly 8.8.x), you would have to apply the attached patch. It's just workaround though, because it has some security implications mentioned by Wim Leers . But that's not enough, you also would have to write custom normalizers for Layout elements. It is what was attempted by #24 . There are more issues if you want to use this exporting capability to later import these nodes with layouts. For example exported references to revision ids of inline block contents in layouts configurations won't match with the real ones, so there is more logic required to get proper references on import. I managed to make it work for my use case, and assuming the attached patch has been applied - it can be done by contrib module.
Comment #55
thursday_bw commentedLuke Nuke, there is a work around for exporting layouts that works now, without any patch by taking advantage of layout builder library module. https://www.drupal.org/project/layout_library
With this you can create a layout in the library, and then instead of adding a layout override on a node, you tell that node to use the layout from the library.. and then it all exports nicely.
Comment #56
geo commentedReworked patches from #52 for D8.7.5
As mentioned in #52 It is a temporary solution. I've created it for using it along with default content export.
Comment #57
wim leersJust be aware that by doing this, you're on your own. Layout Builder does not guarantee that those storage details will remain the same. So if they change, your default content exports will stop working.
Comment #58
geo commentedUpdated my previous patch #56 (I've missed several files).
The patch for D8.7.5 and use it on your own risk.
Comment #59
geo commentedOne more patch for D8.7.5 - still use it on your own risk.
Comment #61
vijaycs85Manually tested the patch (8.7.x HEAD) with
default_contentmodule and the exported node has the layout configuration and node-level override details (as below).Comment #63
lpc commentedTested patch manually on Umami (8.7.8) and multilingual website I am working on (8.7.7). I can confirm that Default content exports layout builder configuration.
Comment #64
vaibhavjainThe patch at #59 works well. I checked with JSON API output, and I am able to see the components there.
However I have a case, where I am not able to fetch the component content.
Question 1 - If we are exposing components, inside the layout, how do we plan to get content of each component ? There are no self or related links, to help you fetch the content and meta data of the component.
Question 2 - If all of the above is true, are we saying that, we can only access content of the block, which are not create via Layout Builder ?
I also see that JSONAPI, wont allow you to access the content of a block which is not reusable. A non Reusable block is created, when you add a block via Layout builder. However, this is also being worked upon here 2999491
Output of my JSON API
Comment #65
vedpareek commentedHi @Vaibhav I was facing the same issue, I have been able to send block bundle and block uuid in json output. This patch has been tested against 8.9.x branch.
Attaching the patch for the same
Thanks
Comment #67
vaibhavjainAs the patch in #65 is incorrect, uploading the correct patch, with interdiff.
Comment #69
zkent commentedHello,
Our website uses a fair number of custom landing pages built using the landing page content type and modified using the layout builder. What is the status of a patch for this issue?
Zach
Comment #70
lpc commentedPatch #59 works well.
Comment #71
aldibierPatch #67 works like a charm! tested in Drupal 8.8.1
Comment #72
lpmthong.dev commentedI used #67, now can see the layout_builder__layout but how can I use jsonapi to PATCH the node to change the layout? Always get errors when tried to PATCH.
Comment #73
vijaycs85Re-rolling against 8.9.x (and works for 8.8.x as well).
Comment #75
arthur.islamov commentedComment #76
vijaycs85@arthur.islamov could you provide interdiff?
Comment #77
morbus iffConfirming #72 is still an issue with #75: When attempting to PATCH a layout_builder__layout field, I receive: "The current user is not allowed to PATCH the selected field (layout_builder__layout)." I receive the same error with POST, meaning I can't create a custom layout in a node (i.e., I can't create custom landing pages from JSON:API). That's my sole need - is this patch only for EXPOSING the data, or also for creating/updating that data?
Comment #78
Oscaner commentedComment #79
i-grou commentedTried patch #78 and it works fine for export, but somehow
$section_data->getValue()(inLayoutSectionItemNormalizer->denormalize()) produces an error when importing node with individual layout (using content_sync module):So to workaround it I've added a check if method exists, if not - then using
$data['section']instead works fine.Maybe it's better to debug
SectionDataclass, but I got no luck with this (in 15 min), maybe someone else can handle it. Or maybe it's related ot my drupal version - 8.8.4. Meanwhile we can go ahead with this patch.Comment #80
i-grou commentedI've ran some more tests and it turned out that patch above not working as expected. Hope it would be fixed in the next release.
Comment #81
aleevasThe latest patch was rerolled
Comment #82
aleevasFixed for latest patch
Comment #84
jonathanjain commentedI have applied #82 patch
Comment #85
adam3145 commentedI just tested this on Drupal 8.8, with layout builder and it's working well with content sync to export my inline blocks!
Anything I can do to help get this commited?
Comment #86
tbsiqueiraCurrently tested for drupal 8.5 works like a charm! Thank you!
EDIT: Thank you for the reply @adam3145 I was using the wrong patch version, I used version 73 and it worked.
Comment #87
adam3145 commentedI am on 8.8.2, I just put the file in the /web folder of my site, and ran the patch -p1
Comment #88
carlosrfm commentedSuccessfully applied patch #82.
Does anyone else is getting this a duplicate "section" tag? This is generating an error whenever I try to import.
Thanks!
Comment #89
adam3145 commentedI am not, do you have multiple sections on your export, as currently I only have 1 on all my pages.
Comment #90
carlosrfm commentedNo, just a single section containing 2 blocks (body and a form).
Comment #91
adam3145 commentedActually yes I do have duplicate sections, however I don't seem to be getting an error when trying to import.
Are you using content sync?
Comment #93
heddnThis hopefully fixes some things.
Comment #94
heddnComment #95
heddnremoves a level of un-needed nesting and general cleanup.
Comment #96
heddnComment #98
heddnHopefully a few more things fixed. The remaining failures I think are more complicated then what time I have to give right now. I did some manual testing and remaining failing tests didn't fail manually (for me).
Comment #100
heddnWe have to remove the uuids as the keys for components since it causes XML serialization to fail. It isn't really needed any way, so I don't think that is an issue. See interdiff for how that looks in umami.
Comment #102
Marko B commentedComment #103
Marko B commentedAs #64 wrote there are important questions this patch doesn't address. Layout builder will and should be used mostly for one time blocks and custom landing pages. So far we get config of the layout but content is non fetchable.
Comment #104
adam3145 commentedI don’t know what you mean content not exposed. I am using this with the content sync module and all of my inline blocks content are exporting and working perfectly
Comment #105
heddnI opened #3151275: referencedEntities should return referenced layout builder added entities earlier to day that might be moderately related.
Comment #106
Marko B commented@adam3145 sorry I didn't make it clear enough. So currently only "reusable" blocks can be fetched with JSON:API, those that dont have that option enabled have access restriction. There is this message "The current user is not allowed to GET the selected resource. Non-reusable blocks must set an access dependency for access control." that comes from getAccessCheckedResourceObject which gets access info from
web/core/modules/block_content/src/BlockContentAccessControlHandler.php and checkAccess which has this
meaning you won't be able to fetch with JSON API non-reusable blocks with current access checker in getAccessCheckedResourceObject.
Comment #107
Marko B commentedOther problem we have is that JSON:API output is not really DX friendly, simplest example would be that we get this export for simple custom block
which you then need to substring to get proper endpoint which would be jsonapi/block_content/hero_block/365dd4c2-41ad-469a-94b3-dc426e8587d0. This looks more like hardcoding stuff then dynamically getting proper endpoints.
And this is simple example. I have much more complex, like having ID from commerce block
How to get proper url for this one :) is it even obtainable in current setup. There is some UUID but which endpoint to use here, JSON output doesnt help much?
Comment #108
tim.plunkett#70 removed the issue summary, not sure why.
Restoring what it was, but I think it needs an update anyway.
Comment #109
geek-merlin#106: This means there will never be access for those one-shot blocks on their own.
I suppose Includes come to the rescue, and maybe #105.
Comment #110
adam3145 commentedThat’s interesting. I am using content sync and my non reusable inline blocks are exporting perfectly. I think I am using patch from 82 or so.
Comment #111
Marko B commentedI am using patch #78 as this is last one that works with 8.8.6 but I don't see this worked on on later patches and mentioned in comments and interfdiffs. If someone can verify this has been worked on later and where, would gladly check it.
Comment #112
Marko B commentedAlso if you are using revisions for your LB and blocks also have revisions you will probably have problems with jsonapi and UUIDs. Seems to me this is the problem https://www.drupal.org/project/drupal/issues/3043321 which leads to this
https://www.drupal.org/project/drupal/issues/3031271
For me block uuids in LB json object dont match what I can fetch, when exposing them they return nothing, but I can manually find proper versions that JA can expose.
Comment #113
adam3145 commentedI have made a module that automatically fixes the UUID's for the purposes of content sync with exposed inline blocks. I can share it here if that's helpfull, it is a stand alone module that I was going to post while this is fixed up in core...
Comment #114
heddnI ran into an issue recently when exporting the layout builder data on a node that the inline blocks are linked via bid/vid. I can't just use their UUID and it relinks them when re-importing the content. That seems related to #112/113, is that true?
Comment #115
adam3145 commentedSo I have a module that i did to fix that, It's attached. You just import normally then run this drush command to clean it up. "drush content-sync-fixer:all"
What we had to do is:
Layout Builder adds a new field to the tables where this feature is enabled, this field is a BLOB field where the layout configuration is saved as a PHP serialized object, each object in the layout is stored in this field with all the proper data to be shown, but, Layout Builder doesn't work as a Drupal entity itself it works in the same way as a text in a field.
Content Sync converts any content in a config presentation in YML when it is exported and then each file is interpreted in a reverse way. Content Sync creates blocks normally when it imports the content, but the relationship stored in the BLOB field is not processed to try to connect properly the content of the field with the blocks. Then the revision_ids inside each inline-block in the layout configuration could not match with the revision_ids over the imported blocks, this is why some inline-blocks are missed and others are misconnected.
To follow up the issue you must query the database in both instances (origin and destination) and check the related data to known entities and fields.
To get specific node information
SELECT * FROM node_field_data WHERE title = 'Web Design';
To get layout builder data using previous information
SELECT bundle, deleted, entity_id, revision_id FROM node__layout_builder__layout WHERE entity_id = 7;
To get serialized data
SELECT CAST(layout_builder__layout_section AS CHAR(50000) CHARACTER SET utf8) FROM node__layout_builder__layout WHERE entity_id = 7;
You will get some like this...
image.png
If you check the layout builder information with the block_content table in both instances (origin and destination), you could check that the information is the same (after import in destination)
But, if you check specific block using
SELECT * FROM block_content WHERE revision_id = 107;
It is possible that block content doesn't match, se below images
image.png
Our solution processes the content inside each node__layout_builder__layout record to connect each inline-block stored in the BLOB field taking in count its UUID code to connect to the correct block.
Comment #116
ilya.no commentedI'm currently using latest patch and noticed, that new options have been added to the configuration, but no corresponding schema. Attaching patch with added schema changes.
Comment #117
alexmoreno commentedI've been testing this with latest patch in #116. TLDR, it works.
Now, the long answer, in case it helps anyone else.
Use case: Drupal as a repository of content for a js application (react, but could be vuejs or anything else).
Layout builder is needed to build the content and expose it via json. A content type with layout builder enabled and a few custom blocks for testing.
Used:
- Drupal Version 8.9.2
- https://www.drupal.org/project/jsonapi_explorer (for testing and exploration purposes)
New brand install, enabled json api and layout builder, left everything else untouched. From there, I created a few landing pages where I am overriding the layout, adding new columns, adding blocks, custom blocks with specific fields, etc…
I am fetching the whole list of nodes with this query:
http://alexplayground.lndo.site:8000/jsonapi/node/landing_page/
Then, once I know the node that I to check out, I just fetch that one, say for example:
http://alexplayground.lndo.site:8000/jsonapi/node/landing_page/?filter[d...
Results:
Without the patch I could not see any information related to the layout or the fields exposed on that layout.
After that, I applied the path and I repeated the query. Then I could see how a new field comes available: layout_builder__layout
On it I can see the different layouts, the components I have created inside those layouts, layout_id, column_widths, the weight of each component, etc.
Something to notice is that there will be no information on that query regarding the fields contained on those blocks, apart of label, uuid, and a few other bits. See:
This can be puzzling on the beginning if you are not familiar with json, and make you think that you are missing information on that query. However, and please correct me here if I'm wrong, I think this is expected. Those extra bits require an extra jsonapi call using the uuid that is provided. For example, I have several blocks on that payload. I choose one of them, get the uuid and make a query like this:
http://alexplayground.lndo.site:8000/jsonapi/block_content/text/d49bdb94...
Comment #118
geek-merlin@alexmoreno #117:
Great writeup!
> Something to notice is that there will be no information on that query regarding the fields contained on those blocks, apart of label, uuid, and a few other bits.
> This can be puzzling on the beginning if you are not familiar with json, and make you think that you are missing information on that query. However, and please correct me here if I'm wrong, I think this is expected. Those extra bits require an extra jsonapi call using the uuid that is provided.
Can you check if Includes do work for this case?
Comment #119
vaibhavjainThank you for explaining things well @alexmoreno. I had same issue (https://www.drupal.org/project/drupal/issues/2942975#comment-13341530), and hence added type and UUID fields to be output (https://www.drupal.org/project/drupal/issues/2942975#comment-13361683). This lets me atleast build the JSON:API URL, which I can use to fetch the content of the components placed, via layout builder.
Yes, this was tested well with custom blocks and multiple block type, however never tried with fields from node or any other blocks from contrib modules.
IMO, we should be providing the relative link automatically, so that we can hit one specific key to fetch the content, than building a URL currently.
@geek-merlin I have verified this before, JSON:API includes does not work well here. Hence the route of adding more info to the output was taken.
Comment #120
geek-merlin> I have verified this before, JSON:API includes does not work well here.
Interesting. We should understand and document why. Or even better, make this work, maybe in a followup.
Comment #121
alexmoreno commented@geek-merlin what I can see is that the document specifies that you need to use "relationship", ie:
then you can make a query like this:
/jsonapi/node/article/some-random-uuid?include=field_comments.uidwhere field_comments is specified on the relationships.
However on my tests I would not get those relationship fields, at least not for the layout builder components. I may not be doing it right though, see payload attached for reference.
https://www.drupal.org/files/issues/2020-07-17/response-layout-builder-p...
Comment #122
geek-merlin@alexmoreno: Thanks a lot!
I did not check the source how JsonApi populates that reference field, but the hottest guess is that it uses EntityInterface::referencedEntities. In which case fixing this at least needs the referenced issue (it may or may not need more).
Comment #123
dmitry.korhovAny news so far? This functionality is pretty essential for decoupled Drupal.
Is https://www.drupal.org/project/drupal/issues/3151275 a blocker?
Can we have this patch released at least as experemental module?
Comment #124
alexmoreno commentedHi again,
I have found an scenario where this is not working. I don't think it's expected, but please keep me humble here
If you override the node with a custom layout, the layout data is exposed correctly (as shown in #117).
However, if you have a default layout, and you don't override it in your node with the aim for that node to use the default one, then the json returns that the layout is empty. See:
```"layout_builder__layout":[]}```
When selecting as well that
Another case, if you specify that you don't want for that content type to be customised by the user on the node level, then the layout_builder__layout does not even appear on the json data.
Does this even make sense? Anyone could raise any light on this?
Thanks.
Comment #125
tedbowre #124
Yes it makes sense. This issue just deals with exposing the stored layout information. Layout Builder stores the layout override data in a field it controls
layout_builder__layout. It stores the default layout in the third party settings information of theLayoutBuilderEntityViewDisplayentity. If a particular bundle does not allow overrides then it does not create thelayout_builder__layoutfield for that bundle.It sounds like what you want is "For any entity using the layout builder give me layout information". This would take more custom code either on the client side making more request or server-side augmenting the data that is stored with the entity when it is returned with JSON API.
Yes this because when you make a JSON API request for the node it only returns the actual data that is stored in the field just like any other field. In the case when there is not an override there is not date stored in this field.
This is because the
layout_builder__layoutis not added until entity display is set to use the layout builder and allow overrides. The field is also deleted if the ability do overrides is removed(you can't do this while there is field data, i.e. any existing overrides.)@see
\Drupal\layout_builder\Entity\LayoutBuilderEntityViewDisplay::addSectionField()and\Drupal\layout_builder\Entity\LayoutBuilderEntityViewDisplay::removeSectionField()if you wanted get the default layout when there is no override you would have to make a JSON API request for
entity_view_displayentity that stores this information in it's third party settings. It would tell if the layout builder is enable and the default layout.Comment #126
alexmoreno commentedI did some work trying to get the default layouts for when the nodes are not overriding it.
Here are my notes, I'll publish this code in a module as well, so it's easier to patch in case anyone has any feedback
https://www.alexmoreno.net/layout-builder-exposing-content-type-default-...
Comment #127
alexmoreno commentedsee as well the code here: https://www.drupal.org/project/lb_default_layout
Can we assume that this patch is tested and RBTC? Are there any concerns left?
Comment #128
tim.plunkettStill needs an issue summary update.
The patch is also failing tests.
Here's a code review.
Whoever updates the patch, please include an interdiff.
Why this change?
This addition isn't documented. It also seems to overlap with the next conditional
Drop the else case, and have
$value = $value->value;?Also why is there all of a sudden a $value->value?
Why this addition? Does this change have test coverage?
Nit: remove needless returns/indents
Whoa, this is a data model change, isn't it?
Comment #129
heddnre #128.6. See #100 for the reason for those changes. It was an issue w/ XML seralization. Not so much for JSON/YAML. But we have to support all, right?
Comment #131
Oscaner commentedBased on #116 patch, no changes, just want to patch successfully on Drupal 8.8.10
Comment #132
mayurjadhav commentedAdded patch for 8.9.x, Kindly Review.
Ref from duplicate issue: https://www.drupal.org/project/drupal/issues/3182960
Comment #133
tim.plunkett@mayurjadhav How does that relate to the patch from #116 that I reviewed in #128?
Or the reroll for 8.8 in #131? Your patch is less than half the size, and doesn't seem to have any test coverage.
As I asked in #128:
Someone please help get this issue back on track! Needs an issue summary update, and also a fresh 9.2.x patch that takes #128 into account.
Comment #134
mayurjadhav commented@tim.plunkett Sorry for the incomplete patch...I was in rush and uploaded patch without reading all comments..I'm working on latest patch which will address all the points mention in #128 along with Test.
Comment #135
Oscaner commentedComment #136
manuel.adan#131 just for the 8.9.x branch
Comment #138
kuhikar commentedHello Team
I have resolved phpcs for 9.1.x and added patch: 2942975_137.patch by combining 2942975-drupal_8_9_x-136.patch.
I have followed below steps for 9.1.x version:
Please do check attached images for the reference.
Comment #139
kuhikar commentedComment #140
kuhikar commentedHello Team,
I have corrected CSpell : failed issue of https://www.drupal.org/pift-ci-job/2105852 for #138. Please do check updated patch (
2942975_140.patch, 2942975_d8_to_d9_140.patch ) for resolving the phpcs errors in Drupal 9.1.x.
Please merge #136 and create 3.x version, after that for 9.1.x, please merge
2942975_140.patch.
2942975_d8_to_d9_140.patch created for Unix Line Ending formats and root directory is in /app folder.
Comment #141
dmitry.korhov@kuhikar please add interdiff
Comment #142
kuhikar commentedHello Team,
Please do check updated patch : 2942975-142-d8_d9.patch for resolving hunk error.
@dmitry.korhov, Please check attached interdiff-136-142.txt.
I have applied #136 patch on D89x and #142 on D91x and after that, I run command to get diff for --core/modules/layout_builder.
Comment #143
kuhikar commentedHello Team,
Currently, we are migrating D8 to D9 and please help to merge patch so we will have 3.x version.
Please do check: Drupal 8 end-of-life on November 2, 2021 : https://www.drupal.org/psa-2021-2021-06-29
Kindly help us to release 3.x version.
Comment #144
kuhikar commentedHello Team,
Any update to release 3.x version of this module? Kindly help us.
@dmitry.korhov, Can you please review patch: 2942975-142-d8_d9.patch
Drupal 8 end-of-life on November 2, 2021 : https://www.drupal.org/psa-2021-2021-06-29
Comment #145
acouch commentedI re-rolled this against 9.3.x
Comment #146
tim.plunkettUnfortunately all of the patches posted since #116 have either lost changes or added new unnecessary changes. Switching to an MR and maybe going to try to address #128...
Still needs an issue summary update.
Comment #148
tim.plunkettNR to see how tests are doing
Comment #149
tim.plunkettOkay I went ahead and made the issue summary updates and feedback changes I'd asked for last year, and resolved the remaining test failures.
This should be ready for review now.
Comment #150
trevorbradley commentedCommit e866ee63 is causing a fault on my system:
[error] InvalidArgumentException: Value assigned to "section" is not a valid section in Drupal\layout_builder\Plugin\DataType\SectionData->setValue() (line 34 of /app/docroot/core/modules/layout_builder/src/Plugin/DataType/SectionData.php).
$value is still being passed in as an array for a custom layout section, removing the array filter from SectionData::setValue() causes it to break.
Comment #151
mkalkbrennerHere's a patch based on MR 1131 and the comment #150.
Comment #152
mkalkbrennerSomething went wrong.
Comment #153
tedbowComment #155
mkalkbrennerI ran the failing tests multiple times locally. The result toggles. Sometimes the tests succeed, sometimes they fail.
Comment #156
tim.plunkettPlease stop posting patches now that we're using the MR. Just push new commits to the branch.
Comment #157
mkalkbrenner@tim.plunkett the patch I posted didn't contain anything new. It just reflects an earlier version of the MR, that doesn't lead to errors in default_content_deploy, just as mentioned in #150.
I uploaded it to have something that could be used deployment piplines already.
Am I allowed to revert your change directly in the MR?
The required change is already discussed above and it seemed that you're not convinced yet.
Comment #158
decipheredI can confirm that the patch from #152 works with Tome and JSON:API whereas the current MR works with JSON:API but not Tome, with the errors reported in #150.
It would be ideal to get the MR synced back to the patch.
Comment #159
BalasaranyaV commentedRebased a patch #152 to merge with Drupal 9.3.x branch.
Comment #161
christian.wiedemann commentedHi all, I try to solve the puzzle to load layout builder inline blocks with JSON:API. My puzzle starts here and after a long weekend I just want to ask if I am on the right trail.
This patch provides sections and components but not the associated block content entities. This is by design. To fetch inline blocks we need a second jsonapi call. @alexmoreno mentioned it in #117. (Thanks for the post, it helps to understand such long issues)
Fetching blocks works right now BUT filter by any query doesn't. There is hard coded resuable=true in TemporaryQueryGuard. So I can't fetch them. This is propably also by design? So I can fetch a block with jsonapi/block_content/text/d49bdb94... but not with jsonapi/block_content/text?filter[id]=d49bdb94... or with any other filter.
So I did not find any solution to fetch multiple non reusable inline blocks in one call. To help me out I created https://www.drupal.org/project/jsonapi_include_lb which uses a simple computed field to provide blocks as entity reference.
Is there a better option?
Comment #163
mkalkbrennerRe-rolled against 9.4.x to get test results
Comment #165
mkalkbrennerI ran the tests locally. The same tests fail. As soon as I enable xdebug and step through \Drupal\rest\Plugin\rest\resource\EntityRessource::get(), the tests succeed. It looks like a timing issue which is strange.
If the error happens and the server returns status code 500 it complains about an illegal character in \Symfony\Component\Serializer\Encoder\XmlEncoder.
But I can't debug it. As already mentioned, the error doesn't happen as soon as xdebug is enabled and I try to step through the process.
Comment #166
mkalkbrennerre-rolled against 9.4.x after conflict caused by commit of #3252872: Use CacheableSupportsMethodInterface for performance improvement in normalizers
Comment #168
mkalkbrennerlike described in #165 and seen as different numbers of failures in #163 and #166, there seems to be a timing problem or an issue with randomized content in the tests.
Comment #169
drupalninja99 commentedI am confused why we went away from this approach which return components as an array of objects: https://www.drupal.org/files/issues/2021-08-25/2942975_145.patch
This is much more useful in Gatsby. I saw at some point that change went away back to a nested object.
Comment #170
drupalninja99 commentedI have the same issue cited in https://www.drupal.org/project/drupal/issues/2942975#comment-13361683 and other places where I can't fetch inline block data from JSON:API like I would say paragraphs. Those JSON:API endpoints only include reusable blocks and the patch from this thread does not include all the block fields in layout_builder__layout. That means I am unable to get all the block data I need on the frontend.
Comment #171
drupalninja99 commentedAfter quite a bit of work/testing I found the solution I needed in project jsonapi_include_lb.
Comment #173
sumitmadan commentedI rebased the MR with 9.4.x branch. I am not able to edit the base branch in the MR. I am not sure if I need any permission for that.
@tim.plunkett can you please update that?
Comment #174
sumitmadan commentedI continued testing the MR on my local. I see two tests are failing which are because of a single reason. The xml format is not rendering properly. The error is following:
The reason is the uuid part inside the component key.
Does anybody have a suggestion on this problem?
Paragraph does it like this and it works fine with paragraph:
Comment #175
tim.plunkett#173, thanks for the rebase. I adjusted the target branch.
Comment #176
mattjones86I have given this MR a try today and found that while it did expose the Layout Builder data, it unfortunately didn't do a great job exposing data for the
inline_blockentities.Within each Layout Builder Section these block are referenced with
block_revision_id, but therelationshipsare not attached to allow modules likeentity_shareto know where to find these.It would be ideal if these could be looked up and appended in a similar method to
EntityReferencefields. I did take a look at updating the MR with this myself, but it looked a little too complex to attempt without guidance.From what I understand, it would involve updating
ResourceObjectNormalizer::serializeField()to handleLayoutSectionItemListfields, along with adding quite a few new methods (e.g.Relationship::createFromLayoutBuilderField()) to find and append the correct relationships.Comment #177
tedbowre #176
I agree what that would be ideal but I think that could be follow-up to this issue. This issue is already 4 years old and adding more scope will just make take longer to get committed.
I don't think we can alter
layout_builder__layoutto add this information because then this would probably cause problems for instance where this serialization was being used for import/export purposes. So it would probably have to be somewhere else in the JSON API response. For that reason I think it adds extra complexity to this issue.Comment #178
vacho commentedThis patch is for core 8.3.x and contains array_values for SectionComponent. To avoid to crash the installation.
Comment #180
antonnaviPatch for core 9.3.x (posted in #178) was updated:
Added exposing of
sectionson request (throughjsonapimodule /jsonapi/page_variant/page_variant) of page variant which was built as a "Layout Builder". See more details here: https://www.drupal.org/project/page_manager/issues/3170526Comment #181
antonnavi@tim.plunkett or @joachim is it possible to add changes:
to the issue branch (to be able to export of sections through the jsonapi module on request like
/jsonapi/page_variant/page_variant).UPD: Added, found how to do it by myself. Sorry for bothering.
Comment #182
vacho commentedStill needs to fix, test issues.
Comment #183
trevorbradley commented[Deleted] My comment is about the layout_builder_st contrib module, not core. Apologies for getting my wires crossed. I'll go make my comment there.
Comment #184
tim.plunkettHiding patches since all work should be in the MR.
Comment #185
drupalninja99 commentedA shot in the dark but I liked the array_values() addition that was questioned in #128. This makes it much easier to query component data from GraphQL. Adding to the layout_builder-rebase_drupal_9_3_x-2942975-159.patch so that you can see what it looks like in comparison.
Comment #187
fpoirier commentedI tried path #185 and #180 and I do get the section data although I only get the components definitions without the block values. Here an example :
I did some debugging and I realize that the `LayoutEntityDisplayNormalizer` is never called which could maybe explain why the inline block values are not showing ?
Comment #188
plachI was testing this and got an error on denormalizaton, this updated patch seems to fix it.
Comment #189
plachAnd this the 9.3.x version.
Comment #190
kriboogh commentedSmall update on #189 to silence the return type error.
Comment #191
ranjith_kumar_k_u commentedComment #193
junaidpvExporting works fine with #191, but it fails with this message when try to import the exported node with layout builder settings:
I noticed the exported layout builder settings part having extra level of "section". Here is the patch fixing that issue. Then it worked with importing node with same layout builder settings.
Comment #196
ilya.no commentedAttaching patch for 9.5.x-dev version. Works locally. Tried to fix tests, but no luck so far. Interdiff may be not correct, since some test files were moved to hal module.
Comment #197
flyke commentedI can confirm that patch #196 also applies to 9.4.1
Comment #198
rassoni commentedTry to fix test failed issues.
Comment #199
rassoni commentedComment #201
christian.wiedemann commentedWith the latest patch #199 enabled layout builder sections are not part of json output of entity_view_display jsonapi call.
e.g:
jsonapi/entity_view_display/entity_view_display/2edbf6ce-3960-4f54-86ac-0703774c9d8fThe reason for this behavior is "Drupal\jsonapi\Normalizer\ResourceObjectNormalizer:196";
I created a follow up issue to remove the unset.
see: https://www.drupal.org/project/drupal/issues/3293908
Comment #202
christian.wiedemann commentedComment #203
tim.plunkettThat was added in #3042198: Add JSON:API integration test for LayoutBuilderEntityViewDisplay (after this issue had begun). I don't know that it needs to be removed separately, could happen here
Comment #204
morbus iffDo the updated patches support the PATCH/POSTing of the layout builder layouts field? Back in #72/75: "When attempting to PATCH a layout_builder__layout field, I receive: "The current user is not allowed to PATCH the selected field (layout_builder__layout)." I receive the same error with POST, meaning I can't create a custom layout in a node (i.e., I can't create custom landing pages from JSON:API). That's my sole need - is this patch only for EXPOSING the data, or also for creating/updating that data?"
Comment #206
yahyaalhamadThe field only displays overridden layouts, it does not work with default layouts.
Comment #207
dmitry.korhovThere is a workaround for it, we managed it by triggering Enhancer (from JSONAPI Extras) manually in some event
Comment #208
nielsaers commentedFor those creating a patch from https://git.drupalcode.org/issue/drupal-2942975/-/tree/2942975-reroll-9.4.x and using the the patch from https://www.drupal.org/project/drupal/issues/2946333. There will be a conflict in applying it.
The patch in 2946333 adds the following line to LayoutBuilderEntityViewDisplayResourceTestBase.php:
Below:
Comment #209
junaidpv#196 used to work for me. But now I also need patch from #3293908-5: Expose Layout Builder Sections to entity view displays (jsonapi/entity_view_display/entity_view_display) to work the importing back the content with the layout builder overriden settings.
Comment #211
daniel.pernold commentedReroll for Drupal 9.4.8
Comment #212
_pratik_Patch is applying now for me.
Comment #213
_utsavsharma commentedFixed CCF of #212.
Please review.
Comment #215
arisenAdded missing files SectionDataNormalizer.php and SectionDataNormalizerTest.php to #213.
Patch applying properly. Please review.
Comment #216
arisenComment #217
arisenFixed CCF of #215. Please review.
Comment #218
arisenComment #220
andypostRe-roll and clean-up for 10.1.x
Comment #221
andypostfix CS
Comment #223
andypostFix return types
Comment #224
andypostand bit more
Comment #227
dmitry.korhov`mixed` return type should not be used, we know what kind of entity|variable is returned.
Comment #228
andypostOne more polishing round
Comment #230
andypoststill not clear why serializer does not return tests for failed LB tests, but here copy-paste clean-up
Comment #232
morbus iffHas anyone gotten this to work with POSTing a new node, with a layout_builder__layout configuration? Whenever I attempt to add a layout builder configuration, with a single views block, I get error "The current user is not allowed to POST the selected field (layout_builder__layout).", even though the current user is an administrative user.
Comment #233
Aaron23 commentedHi Team,
This patch works for me 2942975-196.patch
In addition to that I have used layout library module.
When I create a node and selecting the layout library (chosen any of the template which is created from layout library). In the page api response I'm seeing an empty array,
"layout_builder__layout": []
Just simply once, I go to the layout of the node and save it.. I'm able to see the components
Could anyone help to fix it and possible to provide solution???
Comment #234
kybermanHi, if anybody simply needs to export the "layout_builder__layout" field serialized value using a REST view (e.g. for migration), I created a little formatter with a combination of hook_entity_field_access_alter. Any feedback is welcome, thanks!
Comment #235
andypostAfter SF 6.3 upgrade it will need return types
Comment #237
mkalkbrenner#230 doesn't apply for Drupal 10.1.1. Here's an adjusted version.
Comment #238
patriciacb commentedI rerrolled the patch in #237 because still couldn't be applied to version 10.1.1. I couldn't generate the interdiff file because it throws the typical error with whitespaces that I haven't been able to solve.
Comment #239
patriciacb commentedI rerrolled the patch because it couldn't be applied to version 10.1. I couldn't generate the interdiff file because it throws the typical error with whitespaces that I haven't been able to solve.
Comment #240
sorlov commentedreroll for latest 10.1.x
Comment #241
sorlov commentedComment #242
sorlov commentedComment #243
sorlov commentedreroll for latest 11.x
Comment #244
sorlov commentedComment #245
sorlov commentedComment #246
smustgrave commentedSeems there are a few test failures.
Also see a schema change imagine that will need a post_update and tests.
Comment #247
larowlanAt present there's a security issue with this patch, if you're using it to POST/PATCH
\Drupal\layout_builder\Plugin\Block\InlineBlock::getEntitycallsunserializeonblock_serializedIf that value is provided by untrusted user input over an API, you could open the site up to a gadget chain attack.
I also think that points to a larger issue with this patch in relation to JSON:API, and that's that we don't get relationship support for inline blocks.
I've got some thoughts on that and will take some time to write that up in the next fortnight or so.
Comment #248
wim leersI lost this write-up not once, but twice, because my >5 year old iPad is not able to handle looking at 2 browser tabs if one of them is this one. Kind of understandable though: it's 158 KB of HTML 🫣
I haven't reviewed this issue in ~5 years, but #247 shows that there are still security concerns (see the ones I raised 5 years ago at #33).
👏 @larowlan for the detailed analysis over at https://www.previousnext.com.au/blog/pitchburgh-diaries-decoupled-layout... 👏
I agree with his statements, assuming that it is a hard requirement to be able to truly access sections and components as related resources.
I'm not convinced this is truly a hard requirement though. I do see the need for listing all available section plugins and all available components. Without that, it's impossible for the client to know the possible values. But I don't see the need to address/access section (plugin) instances and component (plugin) instances (meaning: as they are used in a specific entity) directly via JSON:API? 🤔
If we'd drop that requirement:
/jsonapi/<node 4453>/relationships/layout+ ametaon the relationship to express that this is a reference for that component?I bet I'm missing something though, because it's been years since I dug into JSON:API or Layout Builder details 😅 But it's a testament to your excellent write-up, that I was able to articulate this. Still an inkling of hope that this will help get this issue going… 🤞🤓
Comment #249
andypostI bet there's different purpose and requirement additionally to subject of the issue.
Moreover this part already exists - #3231584: DiffArray does not serialize Layout builder section objects causing an error with view display configs managed by layout_builder
(De)serialization is required for
- configuration of entity displays with pre-configured sections #3151275: referencedEntities should return referenced layout builder added entities
- entities for contrib Default_content #3160146: Add a Normalizer and Denormalizer to support Layout Builder and similar to #2956221: Issue with Layout Builder and Serializer normalization
Comment #250
larowlan#248 I think that's worth exploring - hadn't considered using meta.
In the meantime I opened #3391461: Harden the use of unserialize in InlineBlock via allowed classes which is a hard blocker for this.
Comment #251
wmcmillian commentedI've re-rolled this and added this to the SectionDataNormalizer:
This was causing an issue when normalizing/denormalizing entities with other typed data attached to them.
Comment #252
adwivedi008 commentedHii everyone
I am using #239 for D-10.1.7 and along with this patch I am also using https://www.drupal.org/files/issues/2023-07-25/2946333-d10-307.patch
The patch applies clearly but I am facing this error while clearing the cache
PHP Fatal error: Class Drupal\layout_builder\Section contains 1 abstract method and must therefore be declared abstract or implement the remaining methods (JsonSerializable::jsonSerialize) in /web/core/modules/layout_builder/src/Section.php on line 21
Please suggest how to resolve this
Comment #253
adwivedi008 commentedRevised #239 for
PHP Fatal error: Class Drupal\layout_builder\Section contains 1 abstract method and must therefore be declared abstract or implement the remaining methods (JsonSerializable::jsonSerialize) in /web/core/modules/layout_builder/src/Section.php on line 21
After comparing with #251
Comment #254
flyke commentedThe most recent patch I found that applies to Drupal 10.2 is #237 at this moment.
Comment #255
adwivedi008 commentedRerolled patch#237 as it was failing for D-10.1.7
Comment #257
kevinquillen commentedWhere does this issue stand? In my case I am only concerned with GET requests to feed headless instances. With the patch, I do get some layout data, for example:
But no inline block data, or relationships.
Comment #258
kevinquillen commentedWith the patch from 255, and:
I can get managed layout data, overridden entity layout data, and block data in layout builder over JSON:API. However, regions are not in order and neither are blocks, and I think that should be a part of this patch. The data should be stored or returned in the order it is saved. Otherwise this forces consumers to have to assemble it after the fact. It may not be perfect but it is starting to finally seem possible that we can serve headless solutions that use Layout Builder and assemble the data in a framework of choice in a decoupled setup.
I think it is more critical to fix those issues so JSON:API data can be read over GET requests at a minimum first to start empowering decoupled solutions and focus on the rest in other issues.
Comment #259
kevinquillen commentedComment #260
kevinquillen commentedAdding #3226285: Components inside a section should be rendered in order of region and component weight as related.
Comment #261
kevinquillen commentedRe-roll for 10.3.
Comment #262
kevinquillen commentedRe-rolled patch to include new files.
Comment #263
flyke commentedSeems to work or at least apply on a clean 10.3.x
Comment #264
flyke commentedComment deleted because the original reroll patch did work. Just not in my project, probably a conflict with another patch.
Update: it seems to be a conflict with #3049332 Because if I do this in my patches section of my project's composer.json file:
Then this patch works, but the path from #3049332 will not apply.
If I do this in my composer.json:
Then this patch will not apply, but the patch from #3049332 will apply.
Unfortunatly, I need both; I have both problems.
Comment #265
flyke commentedComment #266
kevinquillen commentedNoticed that if you try to load and include layout builder data from a default managed layout, you will get an access denied exception with "The administer node display permission is required". This prevents exporting and attaching default layout data to a response for JSON:API - which you would need to have if the viewed node did not override / add a layout (you'd have to check the content type display and return that).
The access check happens here in
core/modules/jsonapi/src/Access/EntityAccessChecker.php:A default layout with block_content entities has a dependency on the entity view display to which it is attached, which will deny access if the current user does not have 'administer node display' permissions.
For our local POC I worked around this for now by patching in this before
EntityAccessDeniedHttpExceptionis returned, ran out of time investigating last Friday:Comment #267
flyke commentedAdded issue #3049332 as related because its impossible to use this patch and patch from that issue at the same time.
Comment #268
gaurav_manerkar commentedMissing `SectionDataNormalizer` class in #264 patch
Comment #269
artsays commentedLooks like into the latest patch #264 for Drupal Core version 10.3.x
Was missed in creating class two classes SectionDataNormalizer & SectionDataNormalizerTest the result Fatal errors
I've updated the patch 2942975-10.3.x.patch
Comment #270
kevinquillen commentedNote that for Drupal 11 compatibility, the following change has to be made in
SectionDataNormalizer, as$supportedInterfaceOrClasshas been removed:Otherwise, this patch will apply cleanly to Drupal 11 and works. I will try to re-roll this week when I get time.
Comment #271
drupalninja99 commentedThe content looks like it exports correctly but then I get this error on import:
I think the function is here in core/modules/layout_builder/src/Plugin/DataType/SectionData.php:
$value is coming in as an array not instanceof Section
Comment #272
drupalninja99 commentedBleh sorry posted to the wrong issue ignore me
Comment #273
rpearl commentedRe-roll of the patch for Drupal 10.4.x.
Comment #274
elgandoz commentedRe-roll of #273 with @kevinquillen recommendation from #270.
It applies to core 11.1.x (it should still work on 10.4.x).
I'm currently using it to export LB content with Tome, it works fine.
Comment #275
smustgrave commented#3293908: Expose Layout Builder Sections to entity view displays (jsonapi/entity_view_display/entity_view_display) should be covered too
Comment #276
phenaproximaAdding #3532694: Add a command-line utility to export content in YAML format as related, because this issue completely blocks the ability to export Layout Builder data.
Comment #277
mkalkbrennerThis issue also blocks https://www.drupal.org/project/issues/default_content_deploy
Anyway, here's a version of patch #274 that could be applied on top of https://www.drupal.org/project/drupal/issues/3049332#comment-16184349
Comment #278
phenaproximaRemoving the default content export issue, as it has gone in a direction that no longer involves the Serialization module.
Comment #279
kalpanajaiswal commentedComment #280
kalpanajaiswal commentedRevised and upgraded the patch to support Drupal 10.5.8.
Comment #281
vasikeNew patch for D11.3 upgrades
Comment #282
mkalkbrennerSame like in #277, now basing on #281
Comment #283
mkalkbrennerUps, forgot a file. Now, same like in #277, now basing on #281
Comment #284
mkalkbrennerFirst, i forgot a file. Then, the paths were wrong. Again, same like in #277, now basing on #281
Comment #285
alex.skrypnykRe-roll of #274 for 11.3.1
Comment #286
mkalkbrennerComment #290
volegerCreated MR for 11.x branch based on the #286 patch.