Problem/Motivation

Similar to #2699085: Lock Panels IPE when a user has made TempStore changes, the section storage needs to have multiple versions of itself stored in TempStore.
Putting this additional information into getStorageId() would change a lot of other logic in places, as well as all of the URLs.

Proposed resolution

Introduce a separate interface to be able to provide a more granular ID

There is no functional change to Layout Builder at this moment, but this patch blocks the work done in #2946333: Allow synced Layout override Translations: translating labels and inline blocks and enables the benefits of #3004536: Move the Layout Builder UI into an entity form for better integration with other content authoring modules and core features to be realized.

Remaining tasks

N/A

User interface changes

N/A

API changes

Add a new interface (marked @internal) TempStoreIdentifierInterface
This could live upstream and has a follow-up issue referenced via an @todo: #3026957: Provide a generic TempStoreIdentifierInterface that any object can use to better identify itself

SectionStorageBase implements this interface, and all other SectionStorage plugins that do not implement the new interface on their own or via SectionStorageBase will continue to work as expected.

Data model changes

N/A

Release notes snippet

N/A

CommentFileSizeAuthor
#38 3026698-tempstore-38.patch19.59 KBtim.plunkett
#38 3026698-tempstore-38-interdiff.txt1.71 KBtim.plunkett
#37 3026698-tempstore-37-interdiff.txt893 bytestim.plunkett
#37 3026698-tempstore-37.patch19.46 KBtim.plunkett
#32 3026698-tempstore-32-interdiff.txt2.04 KBtim.plunkett
#32 3026698-tempstore-32-PASS.patch19.23 KBtim.plunkett
#32 3026698-tempstore-32-FAIL.patch18.68 KBtim.plunkett
#28 3026698-tempstore-28.patch18.68 KBtim.plunkett
#27 3026698-tempstore-27.patch18.69 KBtim.plunkett
#24 3026698-tempstore-24-interdiff.txt2.42 KBtim.plunkett
#24 3026698-tempstore-24-PASS.patch18.71 KBtim.plunkett
#24 3026698-tempstore-24-FAIL.patch18.71 KBtim.plunkett
#18 3026698-tempstore-18-interdiff.txt926 bytestim.plunkett
#18 3026698-tempstore-18.patch17.74 KBtim.plunkett
#17 3026698-tempstore-17-interdiff.txt9.36 KBtim.plunkett
#17 3026698-tempstore-17-PASS.patch17.66 KBtim.plunkett
#17 3026698-tempstore-17-FAIL.patch15.48 KBtim.plunkett
#9 Screen Shot 2019-02-04 at 7.31.22 PM.png261.77 KBkristen pol
#9 Screen Shot 2019-02-04 at 7.34.22 PM.png185.06 KBkristen pol
#9 Screen Shot 2019-02-04 at 7.31.00 PM.png300.73 KBkristen pol
#9 Screen Shot 2019-02-04 at 7.30.21 PM.png181.78 KBkristen pol
#8 3026698-tempstore-8.patch8.68 KBtim.plunkett
#7 Screen Shot 2019-02-03 at 11.37.18 PM.png203.78 KBkristen pol
#7 Screen Shot 2019-02-03 at 11.37.02 PM.png191.78 KBkristen pol
#7 Screen Shot 2019-02-03 at 11.35.55 PM.png286.06 KBkristen pol
#4 3026698-tempstore-4-interdiff.txt7.77 KBtim.plunkett
#4 3026698-tempstore-4.patch8.62 KBtim.plunkett
#2 3026698-tempstore-2.patch5.52 KBtim.plunkett

Comments

tim.plunkett created an issue. See original summary.

tim.plunkett’s picture

Status: Active » Needs review
StatusFileSize
new5.52 KB
tedbow’s picture

This seems like a good ID and could be used in

  1. +++ b/core/modules/layout_builder/src/LayoutTempstoreRepository.php
    @@ -84,4 +84,23 @@ protected function getTempstore(SectionStorageInterface $section_storage) {
    +   *   A unique string representing the section storage. This should include as
    +   *   much identifying information as possible about this particular storage,
    +   *   including information like the current language.
    

    If we want as much of information as possible should just include now as much information as we have. So add view_mode now even though it won't vary by that now.

    That way if we add that in the future an in process tempstore would still work.

  2. +++ b/core/modules/layout_builder/src/TempStoreIdentifierInterface.php
    @@ -0,0 +1,15 @@
    + * @todo.
    

    Do. Do we add @internal but note that it could be put in another namespace. I could see this being useful in other situations.

tim.plunkett’s picture

Status: Needs work » Needs review
StatusFileSize
new8.62 KB
new7.77 KB
kristen pol’s picture

I reviewed that the Id => Key in the interdiff but it probably makes sense for @tedbow to have a look. Is there any way to manually test this?

tim.plunkett’s picture

Actually the bug you hit in #3028821-7: Layout builder doesn't redirect properly to the translation might be a way to test this...

kristen pol’s picture

Status: Needs review » Needs work
StatusFileSize
new286.06 KB
new191.78 KB
new203.78 KB

I'm still seeing the bug from https://www.drupal.org/project/drupal/issues/3028821#comment-12945393 with patch from #4.

tim.plunkett’s picture

Status: Needs work » Needs review
StatusFileSize
new8.68 KB

I wasn't sure that this would fix that bug completely. I think a better way to test this would be to make (but not save) an override in English, and then make (but not save) an override in any other language, and be sure that changes you make to the temporary copy in one language do not affect the temporary copy for the other language.

Here's a reroll.

kristen pol’s picture

Status: Needs review » Needs work
StatusFileSize
new181.78 KB
new300.73 KB
new185.06 KB
new261.77 KB

Ok, I created but didn't save layout changes in English. When going to French URL as same user then it shows the changes that were made in English (without saving). If I added to French layout and not save and then reload the English URL as same user then I see the French changes.

Back to "Needs work".

tim.plunkett’s picture

Status: Needs work » Needs review

Hmm, this doesn't do anything functionally visible without #3004536: Move the Layout Builder UI into an entity form for better integration with other content authoring modules and core features. Sorry for the confusion. But applying that patch (conflicts only on use statements) allows this to be tested.

Alternately, it can be seen in the key_value_expire table of the DB that the language is applied to the tempstore key...

tim.plunkett’s picture

Priority: Normal » Major
Issue summary: View changes
Issue tags: +blocker

Updated the IS and bumping to major

tedbow’s picture

Status: Needs review » Reviewed & tested by the community

Issue summary looks good.

the problems documented in #9 will be addressed in #2946333: Allow synced Layout override Translations: translating labels and inline blocks which this patch helps

xjm’s picture

Status: Reviewed & tested by the community » Needs work

I think this needs an upgrade path of some sort; when I applied the patch to my site and cleared caches with an existing LB tempstore I got some really weird results (a persistent message about the layout being locked that would not go away no matter how many times I saved the layout as the owner or tried to break the lock as the other user).

xjm’s picture

(FWIW I was testing the patch together with #3025231: Concurrent editing of layouts is very confusing so that the tempstore actually worked in the first place.)

tim.plunkett’s picture

Status: Needs work » Reviewed & tested by the community

This only happens in conjunction with that issue. This is safe to commit on it's own, or should be NW'd if the other one goes in first.

xjm’s picture

Status: Reviewed & tested by the community » Needs review

No, I don't think it's OK on its own either... it's still causing the tempstore data to be effectively lost. STR:

  1. Install 8.7.x HEAD and enable Layout Builder.
  2. Go to admin/structure/types/manage/article/display. Check "Use Layout Builder" and "Allow each content item to have its layout customized." Save.
  3. Create an article node and save it.
  4. Click on the Layout tab for the article node. Add a new two-column section at the top and place a block in it. Do not click save. Note that you can leave the page and come back, or even log out of the site, and your unsaved changes with the two-column section will still be there.
  5. Apply this patch and clear your cache. Go back to the node 1 layout tab. Your unsaved section is lost.

A tempstore is not a cache; it is actual stored data from user input that is supposed to persist (for a week in this case) and cannot be regenerated. So I still think it needs either an upgrade path, or some sort of fallback to allow it to find the old tempstores. Furthermore, even though #3026698: Allow section storage to provide a more granular ID for tempstore is not committed yet, the persistent "break lock" message that's impossible to get rid of does demonstrate that the data integrity problem, which may also be surfaced in other ways we haven't thought of as well, including with contrib or custom code that interacts with this tempstore.

@tim.plunkett contended that we don't need an upgrade path for data in key_value_expire that only persists for 7 days. However, I think we do need to do something for this issue, because it still is a data integrity problem for those 7 days. I'll ask other committers what they think.

tim.plunkett’s picture

Here's an update path and test.

tim.plunkett’s picture

StatusFileSize
new17.74 KB
new926 bytes

Protecting against non-reproducible theoretical issues, but why not.
See #3031130: \Drupal\Core\KeyValueStore\MemoryStorage::rename() erases data if keys match

DatabaseStorage is used during functional tests and in runtime, so it's irrelevant here.

The last submitted patch, 17: 3026698-tempstore-17-FAIL.patch, failed testing. View results

kristen pol’s picture

Should this be manually tested again?

tim.plunkett’s picture

This fix in this issue cannot be manually tested. The update path can: start an override on a nice but do not save, apply the patch, note that your in-progress work is missing, run the DB update, see your work has returned

phenaproxima’s picture

Status: Needs review » Reviewed & tested by the community

Walked through the patch and discussed with @tim.plunkett. I think everything here makes perfect sense. RTBC!

tim.plunkett’s picture

Assigned: Unassigned » tim.plunkett
Status: Reviewed & tested by the community » Needs review
+++ b/core/modules/layout_builder/src/Plugin/SectionStorage/OverridesSectionStorage.php
@@ -164,7 +181,8 @@ public function deriveContextsFromRoute($value, $definition, $name, array $defau
-      $contexts['view_mode'] = new Context(new ContextDefinition('string'), 'full');
+      $view_mode = LayoutBuilderEntityViewDisplay::collectRenderDisplay($entity, 'full')->getMode();
+      $contexts['view_mode'] = new Context(new ContextDefinition('string'), $view_mode);

After discussion with @xjm, this needs both an inline comment and an explicit test

tim.plunkett’s picture

Assigned: tim.plunkett » Unassigned
StatusFileSize
new18.71 KB
new18.71 KB
new2.42 KB
+++ b/core/modules/layout_builder/src/Plugin/SectionStorage/OverridesSectionStorage.php
@@ -181,7 +181,10 @@ public function deriveContextsFromRoute($value, $definition, $name, array $defau
       // @todo Expand to work for all view modes in
       //   https://www.drupal.org/node/2907413.
-      $view_mode = LayoutBuilderEntityViewDisplay::collectRenderDisplay($entity, 'full')->getMode();
+      $view_mode = 'full';
+      // Retrieve the actual view mode from the returned view display as the
+      // requested view mode may not exist and a fallback will be used.
+      $view_mode = LayoutBuilderEntityViewDisplay::collectRenderDisplay($entity, $view_mode)->getMode();

I structured the code like this so that the @todo is clearly only for the workaround line, and the comment with the fix is a separate line.

phenaproxima’s picture

Status: Needs review » Reviewed & tested by the community

Inline comment is helpful and the test makes sense to me. RTBC when Christmas arrives.

The last submitted patch, 24: 3026698-tempstore-24-FAIL.patch, failed testing. View results

tim.plunkett’s picture

StatusFileSize
new18.69 KB

Conflict on a recent commit also adding a post_update hook to the end of the file...
Straight reroll.

tim.plunkett’s picture

StatusFileSize
new18.68 KB

Same thing, but again

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 28: 3026698-tempstore-28.patch, failed testing. View results

tim.plunkett’s picture

Status: Needs work » Reviewed & tested by the community

22:57:43 $ cross-env BABEL_ENV=development node -r dotenv-safe/config -r babel-register ./node_modules/.bin/nightwatch --config ./tests/Drupal/Nightwatch/nightwatch.conf.js
22:57:43 /bin/sh: 1: cross-env: not found
22:57:43 error Command failed with exit code 127.

xjm’s picture

Status: Reviewed & tested by the community » Needs review

So the update path fixes the data integrity problem. My data is safely there after I run update.php.

However, the "You have unsaved changes" message disappears from in-progress overrides.

For more fun, when this is combined with #3025231: Concurrent editing of layouts is very confusing, there are no warnings about unsaved changes nor any "break this lock" messages. So essentially, when someone updates to 8.7.0 with both these changes, the fix for #3025231: Concurrent editing of layouts is very confusing takes seven days to take effect. 😂

@tim.plunkett is looking into this. Setting NR for discussion.

tim.plunkett’s picture

+++ b/core/modules/layout_builder/src/Plugin/SectionStorage/OverridesSectionStorage.php
@@ -164,7 +181,11 @@ public function deriveContextsFromRoute($value, $definition, $name, array $defau
+      // Retrieve the actual view mode from the returned view display as the
+      // requested view mode may not exist and a fallback will be used.
+      $view_mode = LayoutBuilderEntityViewDisplay::collectRenderDisplay($entity, $view_mode)->getMode();

This line (plus the update path) is what prevents the data loss.
LB assumes full, but in a stock Drupal install the view mode will actually be default.

However, this needs to be reflected within the temporary section storage itself. Otherwise we get the correct data, but the UI doesn't recognize the temporary override and doesn't show the message.

Since one of the changes was to a file git considers "binary", this isn't visible in the patch.
I changed
s:7:"default";
to
s:4:"full";
for the tempstore's section storage view mode. This represents the mismatch that @xjm was hitting (and that I was able to reproduce).

The FAIL patch is the previous patch plus that fixture change without the change in the interdiff.

The last submitted patch, 32: 3026698-tempstore-32-FAIL.patch, failed testing. View results

Status: Needs review » Needs work

The last submitted patch, 32: 3026698-tempstore-32-PASS.patch, failed testing. View results

tim.plunkett’s picture

Status: Needs work » Needs review

FAIL patch worked as expected:

1) Drupal\Tests\layout_builder\Functional\Update\TempstoreKeyUpdatePathTest::testRunUpdates
Behat\Mink\Exception\ResponseTextException: The text "You have unsaved changes." was not found anywhere in the text of the current page.

Random OffCanvas fail on PATCH, retesting.

xjm’s picture

  1. +++ b/core/modules/layout_builder/src/LayoutTempstoreRepository.php
    @@ -84,4 +84,23 @@ protected function getTempstore(SectionStorageInterface $section_storage) {
    +  protected function getKey(SectionStorageInterface $section_storage) {
    +    if ($section_storage instanceof TempStoreIdentifierInterface) {
    +      return $section_storage->getTempstoreKey();
    +    }
    +
    +    return $section_storage->getStorageId();
    
    +++ b/core/modules/layout_builder/src/Plugin/SectionStorage/SectionStorageBase.php
    @@ -16,7 +17,7 @@
    -abstract class SectionStorageBase extends ContextAwarePluginBase implements SectionStorageInterface {
    +abstract class SectionStorageBase extends ContextAwarePluginBase implements SectionStorageInterface, TempStoreIdentifierInterface {
    
    @@ -108,4 +109,11 @@ public function getContextsDuringPreview() {
    +  /**
    +   * {@inheritdoc}
    +   */
    +  public function getTempstoreKey() {
    +    return $this->getStorageId();
    +  }
    

    Good BC all around here.

  2. +++ b/core/modules/layout_builder/src/TempStoreIdentifierInterface.php
    @@ -0,0 +1,22 @@
    + * @internal
    + */
    

    Another "why am I @internal?" here.

tim.plunkett’s picture

StatusFileSize
new19.46 KB
new893 bytes

Fixed #37.2

tim.plunkett’s picture

StatusFileSize
new1.71 KB
new19.59 KB

@tedbow ran into an issue when you start using Layout Builder after updating to the new code but before running update.php
He, @xjm, and I collaborated on a fix.
I attempted to write coverage for this, but it is not really possible. We agreed that the fix was sufficient and thoroughly manually tested.

Interdiff was made with -w for reviewability

The last submitted patch, 37: 3026698-tempstore-37.patch, failed testing. View results

The last submitted patch, 32: 3026698-tempstore-32-PASS.patch, failed testing. View results

tedbow’s picture

Status: Needs review » Reviewed & tested by the community

This looks good. I manually tested it and it works good now.

  • xjm committed 589519e on 8.7.x
    Issue #3026698 by tim.plunkett, Kristen Pol, xjm, tedbow: Allow section...
xjm’s picture

Status: Reviewed & tested by the community » Fixed

Committed and pushed to 8.7.x. Thanks!

tacituseu’s picture

Test introduced here hit something on PHP 7.3.x:
https://www.drupal.org/pift-ci-job/1201988
https://www.drupal.org/pift-ci-job/1202908

Drupal\Tests\layout_builder\Functional\Update\TempstoreKeyUpdatePathTest::testRunUpdates
Exception: Notice: unserialize(): Error at offset 7543 of 10758 bytes
Drupal\Component\Serialization\PhpSerialize::decode()() (Line: 21)


/var/www/html/core/lib/Drupal/Core/Test/HttpClientMiddleware/TestHttpClientMiddleware.php:51
/var/www/html/vendor/guzzlehttp/promises/src/Promise.php:203
/var/www/html/vendor/guzzlehttp/promises/src/Promise.php:156
/var/www/html/vendor/guzzlehttp/promises/src/TaskQueue.php:47
/var/www/html/vendor/guzzlehttp/promises/src/Promise.php:246
/var/www/html/vendor/guzzlehttp/promises/src/Promise.php:223
/var/www/html/vendor/guzzlehttp/promises/src/Promise.php:267
/var/www/html/vendor/guzzlehttp/promises/src/Promise.php:225
/var/www/html/vendor/guzzlehttp/promises/src/Promise.php:62
/var/www/html/vendor/guzzlehttp/guzzle/src/Client.php:131
/var/www/html/vendor/fabpot/goutte/Goutte/Client.php:180
/var/www/html/vendor/symfony/browser-kit/Client.php:312
/var/www/html/vendor/behat/mink-browserkit-driver/src/BrowserKitDriver.php:144
/var/www/html/vendor/behat/mink/src/Session.php:148
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:326
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:532
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:333
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:532
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:333
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:532
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:333
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:532
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:333
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:532
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:333
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:532
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:333
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:532
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:333
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:532
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:333
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:532
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:333
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:532
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:333
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:532
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:333
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:532
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:333
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:532
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:333
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:532
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:333
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:532
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:333
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:532
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:333
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:532
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:333
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:532
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:333
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:532
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:333
/var/www/html/core/tests/Drupal/Tests/UiHelperTrait.php:532
/var/www/html/core/tests/Drupal/FunctionalTests/Update/UpdatePathTestBase.php:297
/var/www/html/core/modules/layout_builder/tests/src/Functional/Update/TempstoreKeyUpdatePathTest.php:34
tim.plunkett’s picture

Looks like a bug in 7.3: https://bugs.php.net/bug.php?id=77302

Symfony hit it as well: https://github.com/symfony/symfony/issues/29459

Seems there was a problem with the way I manipulated the serialized string.
calling serialize(unserialize($string)) on it has changed the value.

Opened #3033691: Serialized Layout Builder fixture is invalid in PHP 7.3 to fix this

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.