Prove that it *will* be possible to apply block settings update paths (assuming #3521221 in core) to stored XB component trees in config/content

Plan articulated at #3520484: [META] Production-ready ComponentSource plugins.

Will discuss #8 + #9 tomorrow with @longwave :)

Thanks to #3457504: XB field type: calculate all dependencies, store them, surface in new Component "Audit" operation + #3521202: Store XB field type's "deps_*" columns in separate table to allow efficient querying, it's now trivial to find all the content entity revisions (and config entities) in which a certain block plugin is used in an XB component tree.

But what's still missing is fundamental core support for generically updating the inputs expected by plugins that may be used in multiple contexts, and not just e.g. Block config entities. The core issue for that is #3521221: Block plugins need to be able to update their settings in multiple different storage implementations.

Thanks, @catch, for opening that, and stating it as starkly as you did: Fixing it requires adding a new API to core, don't see another way — I don't either. 😬

Self-assigned in #10, forgot to link here to the write-up I posted at #3520484-22: [META] Production-ready ComponentSource plugins as promised. Unassigning.

Once #3468272: Store the ComponentTreeStructure field property one row per component instance is in, I think we should be able to create a working PoC here that does the following in a test:

1. Create a new input_schema_change_poc block plugin in the xb_test_block module which has e.g. ['foo' => 'bar'] as its default configuration, and has this as its config schema:
   

   block.settings.input_schema_change_poc:
     type: block_settings
     mapping:
       foo:
         type: string
         constraints:
           Choice: [bar, baz]
     constraints:
       FullyValidatable: ~
   

2. Creates a component tree using that
3. Installs a xb_test_block_simulate_input_schema_change module that:
   • overrides which class is used for that block plugin, and changes the default configuration to ['foo' => 'bar', 'change' => 'is scary'], plus changes its render logic in ::build() so we can easily assert the difference
   • alters the config schema to be:
     

     block.settings.input_schema_change_poc:
       type: block_settings
       mapping:
         foo:
           type: string
           constraints:
             Choice: [bar, baz]
         change:
           type: string
           constraints:
             Choice: ['is scary', 'is necessary']
       constraints:
         FullyValidatable: ~
     

4. Make the test verify that the existing component instance renders the message provided by RenderSafeComponentContainer
5. Then write manual logic because #3521221: Block plugins need to be able to update their settings in multiple different storage implementations does not exist yet that:
   • uses \Drupal\experience_builder\Audit\ComponentAudit to find all instances, in both config and content
   • update all instances of block.xb_test_block_simulate_input_schema_change in all component trees across both content and config
   • verify all those entities are again/still valid
   • verify that all those component trees render correctly again 👍

👆 That would allow us to be as confident as we can that we'll be able to do #3524751: [later phase] Component Source plugins: generalized support for schema changes of explicit inputs in the refactored server-side data model 👍

Just did a video call with @isholgueras to unblock him on next steps and clarify the unclear bits in my outline at #15 (🙈 Sorry!).

Updated issue title + summary to convey the intended scope.

Explain relation to SDC and code components.

Fix HTML 🙈

Reviewed and … nice progress here! 👍😄

Steps 4 and 5 in the plan I articulated in #15 are not yet done.

Step 4 is not yet done due to a bug in the test coverage, once that's fixed, that'll force #4 😇

FYI: #3523841: Versioned Component config entities (SDC, JS: prop_field_definitions, block: default_setting, all: slots for fallback) + component instances refer to versions ⇒ less data to store per XB field row just landed, and ideally the test coverage for #15.5 would live in ComponentInputsEvolutionTest. That already contains testStorablePropShapeChanges(), this MR would add testInputSchemaChanges(). 🙏

@larowlan in #26: thanks, much appreciated! 🙏

I kicked off a deep review and pushed a few small fixes/improvements which revealed test weaknesses, @f.mazeikis will continue 👍

P.S.: The 3 new test methods added to BlockComponentTest should be moved as I described in #24. 🙏 This observation underlines that.

Per @larowlan's review remarks 1 and 2, let's land #3526127: Ensure deterministic config export order of config-defined component trees first.

#3526127: Ensure deterministic config export order of config-defined component trees is in :)

Actually … I'm pretty sure that @larowlan meant to link to #3528362: Deterministic Component version hash should depend not only on source-specific settings, but also slots + explicit input schema instead of #3526127: Ensure deterministic config export order of config-defined component trees 😅

I found some pretty serious flaws in the test coverage: 1, 2, 3, 4 and most importantly: this was indeed asserting various edge cases/expectations wrt updates to components in the block ComponentSource, but was not yet testing the very purpose of this issue as described in #15.5:proving that a future automated update path will be possible once #3521221 in core adds the necessary infra.

To be fair: this issue/MR is very abstract and is about ensuring something will be possible, which makes it much harder to write tests 😬. But it's really important that the tests introduced here A) are super clear, B) inspire confidence that we'll be able to support this later without BC breaks to the data model/storage (see #3520449: [META] Production-ready data storage).

Will push this over the finish line. Already addressed ~half of my concerns, other half to follow.

See the remaining open MR threads — they provide pointers and context for the most crucial test coverage changes/enhancements.

The plan I outlined in #15 is now fully implemented.

(Note: some of the test clarity improvements I added today were impossible until #3523841: Versioned Component config entities (SDC, JS: prop_field_definitions, block: default_setting, all: slots for fallback) + component instances refer to versions ⇒ less data to store per XB field row last week!)

Next steps are:

1. upstream infrastructure: #3521221: Block plugins need to be able to update their settings in multiple different storage implementations
2. expand the ComponentAudit service in #3524751: [later phase] Component Source plugins: generalized support for schema changes of explicit inputs
3. add component revision support to the ComponentAudit service in #3524751: [later phase] Component Source plugins: generalized support for schema changes of explicit inputs and/or #3528499: Revisit storage of dependencies in separate table now we have separate deltas per component + remove plugin dependencies + make component version usages auditable

This apparently introduced 2 test failures on all DBs except SQLite: https://git.drupalcode.org/project/experience_builder/-/pipelines/519184 — JsComponentTest::testFallback() and SingleDirectoryComponentTest::testFallback().

This makes it pass again:

diff --git a/src/Plugin/Field/FieldType/ComponentTreeItem.php b/src/Plugin/Field/FieldType/ComponentTreeItem.php
index 93df89230..547a5b3b0 100644
--- a/src/Plugin/Field/FieldType/ComponentTreeItem.php
+++ b/src/Plugin/Field/FieldType/ComponentTreeItem.php
@@ -393,7 +393,6 @@ class ComponentTreeItem extends FieldItemBase {
     }
     $pairs = [
       ['component_id', 'component'],
-      ['component_version', 'component'],
       ['parent_uuid', 'parent_item'],
     ];
     foreach ($pairs as $pair) {

… but that's a necessary change. I bet @larowlan will figure this out in no time 🤓

Clearly I misunderstood how “pairs” worked 😅

Thanks!