Problem/Motivation
- Layout Builder doesn't support translatable layouts but we still have the "Layout" tab when viewing an entity translation. In 8.6.x translations do not work at all. After #3004536: Move the Layout Builder UI into an entity form for better integration with other content authoring modules and core features there was sort broken support for translations but this was not backported to 8.7.x.
- We also need to make sure that
@FieldType=layout_sectionis not translatable.
Here is the problem that you could get translated, but not updatable, layout overrides in 8.6.x .
- The layout tab for overrides always showed and edited the default language layout even on a translations layout tab
- The only way to get a layout override into the non-default language field value was to create an override before you create a translation. Then the layout would be copied over on creating the translation
- In that case the view of the translation would show the copied over sections but the layout builder would continue to show/edit the default language layout.
- The non-default translation entity view would be stuck with a layout you could never change.
- if the user never went to the layout tab for the translation and after this only went to the layout tab for the default translation then the user may not have every known they could not translate the layout further.
- The user could continue to alter the layout for the default language
- Given the last 2 points the user may have actually not known they had an unchangable translated layout override and from their perspective they were able to translate the layout because they had different layouts for each language
In addition to this translations created when the untranslated entity had no layout override would always show the default layout configuration for the bundle. The user may have perceived this behavior to be the ability have a layout override only for the default language.
If we updated all layout fields to non-translatable then:
- When the user viewed the entity translations Drupal would only load the default translation layout so they would see a different layout than before the update.
- Furthermore if they looked at their revision history of a translation they would also see the default translation layout. So their revision history would look different
- The translated layout sections would still exist in the database until the user edited the translation(not the layout which is not possible). At that point the translation sections would be deleted/cleaned up because it was no longer a translated field
- Also entity translations that had no translated layouts but the untranslated entity had a layout override would switch from using the bundle default layout to the untranslated layout override.
Because of the above we cannot set any layout fields to untranslatable because different entities could be in different states and different users depending on if they ever tried to edit a translated layout may not perceive the current behavior to be a bug. But we also cannot allow them to update the translated layouts because we don't actually support that.
Users may be relying on the ability to configure the default translation layout override and have the translations be unaffected(even though this was not a intentional feature)
Proposed resolution
Now that #2946333: Allow synced Layout override Translations: translating labels and inline blocks has been postponed we now need to take a few steps to make sure translation support is not exposed to users.
- Ensure that the "Layouts" tab is not accessible on entity translations.
- Any new layout fields added should be untranslatable.
In the Change Record we need to explain that to obtain the current module functionality for overrides for existing layout fields the must be manually set to non-translatable. It also details the implications of doing that.
We will not put a message in the UI either on the Content Translation configuration page or on the status report page because there is no way to know whether the field is translatable because the module was installed before this issue or if the field is translatable intentionally because they have a contrib module or custom code that uses the translatable fields to actually implement translations.
Remaining tasks
fixes, tests, review
User interface changes
No "Layout" tab when viewing entity translations.
API changes
None.
Data model changes
All @FieldType=layout_section instances are marked as untranslatable.
Release notes snippet
Layout Builder overrides are now stored in non-translatable fields. This means that on entities with overridable layouts, the Layout tab is only available when viewing the entity in its default language (not from a translation) and layout changes made from there apply to all translations. For sites that installed Layout Builder and enabled layout overrides for some entity bundles while it was still Experimental (prior to 8.7.0-beta1), this release retains the site's existing translatability configuration of those fields. For those sites, manually setting the layout field to non-translatable is strongly recommended unless there is existing layout translation data. See the change record for more details.
| Comment | File | Size | Author |
|---|---|---|---|
| #81 | 3041659-translation-81.patch | 21.9 KB | tim.plunkett |
| #62 | 3041659-translation-62-interdiff.txt | 1.82 KB | tim.plunkett |
| #62 | 3041659-translation-62.patch | 22.08 KB | tim.plunkett |
| #61 | 3041659-translation-61-interdiff.txt | 2.48 KB | tim.plunkett |
| #61 | 3041659-translation-61.patch | 21.42 KB | tim.plunkett |
Comments
Comment #2
tim.plunkettTagging
Comment #3
tedbowHere is patch with tests.
I did this because 8.6.x has a problem if you add layout override first because when you create the translation the layout value are copied over. but then you can never change the translated layout because the UI only affects the untranslated layout field value.
Should we attempt to make the field untranslatable? Will this remove all translated field values?
Otherwise if we don't want change existing fields we still need make sure that fact they have the layout field as translatable doesn't cause problems with new entities and translations.
To do this we would have to implement
hook_entity_translation_createand make sure when a translation is created the layout field values are copied over. Otherwise the translation would always use the layout override at the time the translation was created.Comment #5
tedbowRemove test that depended on access the layout test on translations.
Comment #6
tedbowwhoops here is the patch
Comment #7
plachCan we add an inline comment to explain that we do not support translations yet and maybe a @todo pointing to #2946333: Allow synced Layout override Translations: translating labels and inline blocks?
No need for
canonicalhere.I guess we should restore the previous user after the translation has been created.
Comment #8
plach[duplicate comment]
Comment #9
tedbow@plach thanks for both these reviews 😜
I talked with @plach about what to do about existing sites with override.
First the behavior in 8.6.x for translations:
The idea we had for the new update hook would be to set all existing layout fields as non-translatable.
So the update itself would not remove any data from the database but the next entity save could if a new revision was not made.
We could add in the Change Record for this issue to mention this and that if somehow a site had got translation overrides to work in 8.6.x they would immediately have to set the field to translatable again after the update. Of course they would also have to probably update whatever custom code they had that get to override translations to work correctly in 8.6.x because this same code would almost for sure not work in 8.7.x.
Also of course there would be no guarantee that whatever custom way that a site got translated layout overrides to work would be compatible with the way that core will eventually implement for layout override translations in 8.8.0(or whenever it happens).
Comment #10
tim.plunkettHere's an update path test.
It passes by itself, and fails without the update path.
Once the update path is written, it should fail with that but without the fix, and pass all combined.
FAIL patch is the interdiff.
Comment #13
tim.plunkettAt least they were different fails...
Looking into it
Comment #14
plachWhy NULL? These will be 1 if you create a new translation while updating the default revision. It likely doesn't matter but better safe than sorry :)
Why do we need this?
Ideally we would do
$node->getTranslation('es')->toUrl(), but we'd need to reload the node first...Comment #15
tedbow@plach thanks for the review. I didn't get chance to address in this patch
This patch does
hook_entity_translation_createComment #17
wim leersClarifying.
Comment #18
wim leersLooks like a "layout" tab is available when viewing entity translations. This appears to be a regression compared to #10.
I suspect this is the change that introduced that. I think it's missing a negation.
Here's a patch to fix that.EDIT: oh wait, I think that this is an intentional change, per #15.3 and this addition in #15:
— why this sudden change of direction tough? I don't see this mentioned in the issue summary or the comments here to do this?
Comment #19
tedbow@Wim Leers sorry for not explaining. Updated the summary.
This was an attempt at humor, wit, facetiousness, jocularity 😜
Comment #20
tedbow\Drupal\Tests\layout_builder\Kernel\OverridesSectionStorageTest::testAccess(). To do this I had to actually check$default_section_storage->isOverridable()inOverridesSectionStorage::access()otherwise the field would not be there.Comment #21
wim leers"could NOT set it", right?
Let's also document how that can happen? Right now it sounds rather mysterious. Ominous even.
Comment #22
plachReview of #15:
mmh
Nit:
!$entity_type_has_translated_layouts?"translated layoutS"?
We're missing a condition on the
langcodecolumn being the same (langcode is an entity key, so in the data table the actual column name might be different).Shouldn't this be TRUE? If there is no data table the can't be any translation.
default_langcodeis an entity key.We should add a
dev-speaktranslation on localize.drupal.org so we don't lose this ;)(#14 is still TODO)
Comment #23
plachComment #24
tedbowre #14 @plach thanks for the review.
1 other change.
We actually want the layout link if there were translated values
so this patch passes fails but it passes if I revert #2554235: Make the content entity storage and entity query use the last installed definitions instead of the ones living in code. @plach you worked on that so hoping you my have some idea why.
If I print the test results this is the error on the node view.
@tim.plunkett helped me debug this and he suggested in coming from
\Drupal\Core\Entity\ContentEntityStorageBase::getLatestTranslationAffectedRevisionId()Where 1 of these keys is returing. Guess it is revision_translation_affected because was added later in
system_update_8402Comment #26
amateescu commentedI looked into the failing test and the problem is that
system_update_8402()only added the base field definition but it didn't update the entity type definition as well to add the entity key. We need a separate issue to fix that..While I was here, I cleaned up the update helper function and a few coding standards issues.
Not sending this patch to the testbot because it will have the same fails.
Comment #27
wim leersTalked to @amateescu in Slack, he's opening an issue tonight to fix the problem he discovered in #26, when he's back at his laptop. He knows exactly what needs to change obviously, so having him do it would result in the clearest and hence easiest to commit issue+patch 🙂
Comment #28
tedbow@amateescu thanks for looking into this problem!
Sorry I forget to mention this is a previous comment but pretty sure EntityQuery won't work in this case.
@plach and I spent time debugging this with a previous version that used an EntityQuery and the problem was it was not joining the tables with the 'langcode' key condition.
Basically if you have 2 rows in
node_field_revisionfor the same revision because there is the default language and 1 translation. But innode_revision__layout_builder__layoutyou may only have 1 row for therevision_idbecause only the default language has a layout.Without the join on
langcodethe query will still return 1 result even though there is condition ondefault_langcode = 0which the row innode_revision__layout_builder__layoutdoes not satisfy.For layout builder you can create this situation in 8.6.x by creating a translation first and then creating a layout override for the default language.
I have tested manually to be sure I am correct but currently the fixture
layout-builder-translation.phpdoes not cover this case.I will update that fixture to include another node of different content type where it has a translation but not a translated layout.
Then we will have 2 different content types 1 where the field should be left as translatable because there is translated layout and another where it can be safely set as non-translatable.
Comment #29
amateescu commentedHere's the issue that fixes the missing
revision_translation_affectedentity key: #3042993: Translatable and revisonable installed entity type definitions are missing the 'revision_translation_affected' entity key, and a combined patch with #26.I think the problem you described in #28 is tracked over here #2933202: EntityQuery returns incorrect results for notExists condition where one translation matches but others do not, I'll try to reproduce this scenario locally and see if we can tweak the entity query to work around that bug :)
Comment #30
tedbow@amateescu thanks for creating the patch for #3042993: Translatable and revisonable installed entity type definitions are missing the 'revision_translation_affected' entity key!!!!!
This patch
_layout_builder_no_translated_layouts()not use the entity query because of #2933202: EntityQuery returns incorrect results for notExists condition where one translation matches but others do nottestDisableTranslationOnLayouts()to check Basic Page doesn't have the layouts tab.Comment #33
tedbowTrying to debug this by looking at printer output it was clear which bundle the page was on. Adding bundle the title.
This fails in the subdirectory on Drupalci. Since we are still on the default translation at this point we can just click the links "Translate" and the the Spanish title
We should not be checking this here as we checking it based on
$translated_layout_expectedbelowComment #34
plachOk, we should really come up with something final now :)
What about "Layout builder does not support layout localization, read the documentation for more details."?
This need to be 1, see #14.
Nit: "canonical" is not needed here.
Can we restore the previous user after creating the translation to perform a more consistent test?
Can we add an assertion also for the storage definition?
Is this needed?
Can we add an assertion to check that the warning message is displayed?
Comment #35
tedbowre #34
@plach thanks again for the review
// Create a translation.comment. because the name of the method isaddEntityTranslation()and we have the docblock too.Change to
_layout_builder_no_translated_layouts()I realized that to ensure that we don't change the output of existing sites entities and past revision we can't just check if there are entities with translated layouts.
Actually we have to check if there are any entities for the bundle that have both translations and layout overrides, regardless of whether the layout overrides are for the translations.
This is because in 8.6.x if you created a layout override for the default language after you created a translation then the translation would have no layout override translation(and couldn't make one). But this does not mean that entity view of the translation would use the default translations layout override. This actually means that because the layout override storage would have the translated entity and the field is translated then
OverridesSectionStorage::isOverridden()would return false and the translation would use the bundle's default layout.If we made the field untranslatable in this case then after the update the translation would go from showing the bundle's default layout to the layout override which would now apply to all languages.(it would be good if someone else could also manually confirm this)
I renamed
_layout_builder_no_translated_layouts()to_layout_builder_no_entities_with_layouts_and_translations(). I changed the query to find any entities with both an override and a translation regardless of whether the layout override was for a translation. This is actually a slightly simpler query.I also change the test assertion in
MakeLayoutUntranslatableUpdatePathTest::testDisableTranslationOnLayouts()for the basic page content type from$this->assertTranslatedLayoutWorkflow('page', FALSE);to
$this->assertTranslatedLayoutWorkflow('page', TRUE);Because I set up Basic Page content type in #33 to have a layout override for the default language and a translation(but no translated override). I thought this should mean the field for basic page should set non-translatable but as explained above it should not. So no actual changes need to the fixture.
Comment #36
plach@tedbow, #35:
1: Because different layouts per language is localization and not translation, but nbd: users might not be able to tell the difference.
5: Rrright
7: The string would be translated only if the test enabled the
localemodule, so asserting the English string should be fine.I was wondering whether it wouldn't be better to switch to a
LIMIT 0,1ranged query to improve performance on very large data sets.Missing closing quote in the link tag, also we normally use double quotes for HTML attributes.
Comment #37
tedbowAdded
Comment #38
plachComment #40
effulgentsia commentedI haven't reviewed the whole patch yet, but just some drive-by nits of the post_update:
Any reason not to use the
OverridesSectionStorage::FIELD_NAMEconstant in the first two lines as well?If "translated layout" is an accurate description, per the docs, then can we also change the function name to
_layout_builder_no_entities_with_translated_layouts()? I think the current name of "layouts_and_translations" is inaccurate, because I think a site can have entities which have both layout overrides (in the default language) and translations (without a translation of the layout), and for such a site, this function returns FALSE. Is that correct?Comment #41
tedbow#37 failed because
This failed because of the test is in subdirectory
#40
@effulgentsia thanks for the review fixed.
Comment #42
tedbowJust a reroll
Comment #43
tedbowI chatted with @effulgentsia and he pointed out that it is not super obvious why we are just joining on entity_id here and not langcode and/revision_id.
Adding a commented he suggested.
Comment #44
plachThis is actually referring to Content Translation, right?
Comment #45
plachNit: double trailing dot
This is not great from a visual standpoint, what about appending it to the operation column instead and adding the
messages messages--warningclasses? See the screenshot below for a preview:The last sentence seems truncated, I guess it meant to be something like:
"the following items."
Comment #46
plachThere is no corresponding use statement for FieldConfig, I was getting the following error on the status report:
Also why not moving this to
layout_builder.install?Comment #47
plachNW for the last error.
Comment #48
effulgentsia commentedHas it already been discussed in this issue why we don't care about matching on revision? For example, if I had layout overrides before having translations. Then I stopped having layout overrides and only after that started having translations, then wouldn't it be safe to return TRUE? But if there are other scenarios where layout overrides and translations across different revisions are problematic, then yes, it would be safer to not join on revision ID. I'm just wondering if we know of such a scenario, or if we're just being extra cautious.
s/translated layouts/translations and layouts/
What about the case where the entity type or bundle isn't translatable? How are we handling that case in post_update? Or is there a reason that we don't have to?
Comment #49
effulgentsia commentedRe #35:
Good catch! So +1 to #41.1. I confirmed that this is the standard behavior for adding translatable fields to content that already has translations. For example, create a node with 2 translations, then add a field to that bundle, the go to
/admin/config/regional/content-languageand make the newly added field translatable. If you now edit the node in its default language and put some value into the new field, it only gets applied to the default language. The existing translation, when both viewed and edited, continues to have that new field be empty. Which is different than when you create a new translation, but I can see why this difference is desirable. Given all that,OverridesSectionStorage::isOverridden()is acting correctly (or at least, consistently with translatable fields in general), so you're right about our post_update needing to query for "layouts and translations" rather than "translated layouts".Comment #50
tedbowself review.
s/section/sections
Two periods
This should be in the install file.
unused
We don't actually give them instructions here. Just link to documentation.
Adding a comment here about why we are checking this access.
#44
fixed.
#45
fixed.
Stopping be back 30 minutes or so.
Comment #51
webchickSoooo… @tim.plunkett, @tedbow, and @effulgentsia walked me through the current patch and… I got … very confused at the direction this patch has taken. I was sitting here believing this would be a 10-line patch to just remove the Layout tab from the non-default translation. :D
I know Layout Builder is beta, and therefore we care about the data integrity here… but this patch seems like it's going WAY over the top on hand-holding and putting permanent code in stable core for what I understand is being an issue that only affects a small portion of sites who used Layout Builder before 8.7.0-beta (when it was experimental) and are also multilingual.
It seems like we should be focusing on making sure this condition doesn't happen to new people (so do what the issue title says, and remove access to the tab), and making sure there's something easy to Google for those early adopters with multilingual sites who are suddenly missing a tab. Normally, it seems like we would just put an entry in the release notes that links to a site builder CR for a bug like this, versus codifying something in our UI for the rest of all of time for a temporary condition.
@tedbow had one concern, which was that if a site ever used Layout Builder before 8.7.0-beta, and makes it multilingual later, they will hit the same problem. For this, we could file a follow-up to add a (small) update path to make sure that if a site doesn't have any translations (apparently that's tricky to figure out), then mark the field untranslatable.
Comment #52
tedbowHere is a patch that only
Comment #53
tim.plunkettI completely agree with the proposal above in #51.
This deserves a code comment
Is there any value in keeping this test but changing the assertions to reflect the new reality? If not, this is fine
I don't quite remember why we made this change.
But doesn't the ->save() still need to live outside the if()?
Comment #54
effulgentsia commentedSo #51 is proposing to limit this patch to only:
It's proposing to not do any of:
I agree with punting on the update path. It's getting too close to beta, and we're still having problems with it. I'd rather provide it when we're confident about it, whether that's for RC, in a patch release, in 8.8, or in a contrib module. Although https://www.drupal.org/core/experimental#beta says that beta module provide update paths, we're dealing with a case where this patch isn't breaking anything by lack of an update path. It's leaving an existing site functioning in the same way that it already was. And someone who wants the better behavior, they can go to the content translation configuration page and uncheck the box.
FWIW, I'm also ok with punting on the UI warnings, especially if a product manager doesn't think they're necessary.
Comment #55
plachTotally +1 on #51 as well, I had the same feeling that this was going too far: the scenarios we were accounting for feel way distant from the desired translation support, at least according from the feedback we were able to gather so far. For that reason I doubt there are many sites that are intentionally relying on the current behavior. Those can alter the tab access control and translation creation logic to restore it, we can even add some code snippets to the CR to explain how to do that.
For sites actually being hurt by this behavior, the following workaround is definitely viable:
And if in a follow-up we can provide an update to make fields untranslatable where there's no risk of changing the site behavior unexpectedly, all the better.
@tim.plunkett, #53:
I'm not sure it's worth keeping that test around: the current logic is all covered by
LayoutBuilderTranslationTest, at least from a functional perspective. I'd rather add a small kernel test to coverlayout_builder_entity_translation_create(). Other than that and the remaining bullets in #53, the patch seems ready to go to me (reroll aside).Comment #56
tedbowrerolled
#53
Added a test for translation creation. The test actually passes even without
layout_builder_entity_translation_create()so I am not sure what is going on . I will update a another patch that should fail but doesn't. Search that patch for "WHY CAN THIS BE DELETED AND TEST STILL PASS??????"Comment #57
tedbowTEST GROUP!!!
Comment #58
tim.plunkett@plach pointed out that the second param of
addTranslationis the key here: in the UI workflow the entity values of the default translation are passed in. Doing that here makes the test work as expected.FAIL patch comments out the one line in layout_builder_entity_translation_create()
Comment #59
plachThis looks great to me!
I have a couple of very minor non-blocking remarks that it would be nice to address if we happen to reroll this patch:
I'd move this logic to the
::setUp()method.This is testing that "Per-language layouts are not supported". I think what we mean with layout translation is translating the layout component textual elements.
Comment #60
plachAlso, we need a few updates to reflect the latest change in direction.
Comment #61
tim.plunkettFixed those, and applied the same change as 2 on the docs for our hook.
Also we need an addition for the release note.
Comment #62
tim.plunkettIn order for custom code to be able to restore translation functionality, splitting the access check out to a protected method.
Comment #64
plachOk, aside from the pending updates this looks RTBC to me.
Comment #65
tedbowComment #66
effulgentsia commentedAdded a release note snippet to the IS.
Comment #67
tim.plunkettplach signed off in #64, pending the IS/CR/RN updates.
Confirming that this is now RTBC.
Thanks!
Comment #68
plachThe IS and snippet look good to me, I performed a couple of small tweaks to the CR.
RTBC +1
Comment #69
xjmComment #70
xjmOK I'm really concerned that the update path was abandoned here. We're knowingly deleting site data for sites that unknowingly started a translation before, with no workaround or other option for those sites to preserve their data.
Comment #71
xjmThe release note says:
Is that still true? It contradicts the patch and recent discussion in the issue. Site owners need to know the impact this will have and data loss/orphaned data is a pretty serious disruption.
Comment #72
xjmFor me, whatever followup issue to fix existing sites is a stable blocker. I've reached out to catch for a second opinion.
Comment #73
xjmAfter reading the CR carefully and discussing with @tedbow and @effulgentsia, I'm really concerned with this part:
I asked @effulgentsia if this meant what I thought it did, and he confirmed:
To me this is very much a data integrity issue. There is no way that sites that started using Layout Builder in 8.6 are going to find this one buried bullet point in the 8.7.0 release notes when they decide to enable Content Translation next year. There's no workaround, no action they can take now. We need either in-context messages about this to the user in the UI, or an upgrade path to fix it.
Comment #74
effulgentsia commentedWhat about committing this patch as-is, and making #73 a stable-blocking follow-up? The patch here doesn't regress anything. Maybe it's just not enough to mark the module stable yet?
Comment #75
xjmTed said he would file a separate issue for the update path. For me, an update path to address at least #73 (update the field on monolingual sites) might be stable-blocking. @effulgentsia also mentioned that part of the problem has been upgrade path tests, but for a data integrity issue like this, I would prefer a thorougly manually tested upgrade path with tests descoped to a critical followup, rather than no upgrade path at all. But we can discuss in the other issue if it does get split out.
Crossposted with #74. I think one of two things is stable blocking: UI messages to the user about what happens when this field is marked translatable, or an upgrade path. I'd be okay with followup issues for both, tagged as stable blockers, but that we could decide to descope following discussion or if one or the other lands.
Comment #76
tim.plunkettI created #3043646: For sites that have made layout overrides prior to 8.7.0 or sites that manually enable translation of the layout override field, add UI warnings and it is linked to this issue.
Comment #77
xjmComment #78
xjmComment #79
xjmI updated the release note to be a bit clearer about not recommending the old behavior, and worked on an update of the CR with @effulgentsia.
@tedbow, @tim.plunkett, @effulgentsia and I discussed #73 and agreed that this can go in on its own, since it mostly solves the data integrity problem for sites that haven't already enabled layout overrides somewhere. However, the two followups #3043646: For sites that have made layout overrides prior to 8.7.0 or sites that manually enable translation of the layout override field, add UI warnings and #3043651: Update the Layout field to non-translatable when possible. remain as stable blockers for the time being, until we can decide how to handle existing overrides as well as sites that might enable translation on the field in the future.
Comment #80
xjmComment #81
tim.plunkett#3042089: Update Layout Builder functional javascript tests now that local tasks are not required by the UI. removed one line in a file we are removing. Straight reroll.
Comment #82
effulgentsia commentedCrediting reviewers.
Comment #85
effulgentsia commentedPushed to 8.8.x and 8.7.x and published the CR. See you in the two followups from #79.
Comment #86
plach+1 on #79, sounds like a very reasonable way forward, thanks!
Comment #88
plach