[META] Production-ready data storage

Legend

• ✅ Done.
• 🟢 Confident it will not require backwards compatibility breaks.
• 🟡 Medium confidence; more work needed to achieve necessary confidence.
• 🔴 Low confidence; requires a lot more work

Overview

XB's data model (ComponentTreeItem) was originally prototyped with:

1. only SDCs
2. ✅ dynamic data fetched from the host entity, the right value(s) extracted using (Reference)Field(Object)PropExpression and assigned to the right component instance into the component tree: DynamicPropSources
3. ✅ static data stored in independent field item objects: StaticPropSources (which also must extract the right value(s), and does so using (Reference)FieldType(Object)PropExpression)
4. ✅ symmetric + asymmetric translation support (symmetric: tree is locked, inputs are per-translation; assymmetric: both are translatable, meaning different component trees for each translation)

Those foundations remained largely the same, but since then we gained:

1. ✅ multiple component types, thanks to ComponentSource plugins ⇒ everything must scale to unknown future component types
2. ✅ additional prop sources, such as DefaultRelativeUrlPropSource ⇒ everything must scale to unknown future prop sources
3. ✅ auto-saving of component trees

Finally, we know there's both known technical debt, and more features that we'll eventually need to support:

1. ✅ debt: full dependency support, throughout all of the above, to allow for reliable updates
2. 🟢 debt: the explicit input schema must be able to evolve, and each ComponentSource may have different needs for that. For example: block plugins may change their settings (and their config schema), SDCs may gain new optional inputs, JS components may gain new required inputs → #3501708: Prove that it *will* be possible to apply block settings update paths (assuming #3521221 in core) to stored XB component trees in config/content + #3523841: Versioned Component config entities (SDC, JS: prop_field_definitions, block: default_setting, all: slots for fallback) + component instances refer to versions ⇒ less data to store per XB field row here — but #3524751: [later phase] Component Source plugins: generalized support for schema changes of explicit inputs later.
3. ✅ debt: we went with a JSON-heavy initial implementation (the inputs and tree props both store a JSON struture), but we may want to reconsider this → solved in #3468272: Store the ComponentTreeStructure field property one row per component instance, with additional refinements possible post-release for more field types than just XB
4. ✅ feature: content entities' XB component trees must be able to populate the exposed slots provided by the corresponding ContentTemplate (→ #3519352: Content templates, part 3b: store exposed slot subtrees on individual entities)
5. 🟢 feature: asymmetric translation will need some explicit inputs to be locked, because some explicit inputs may be considered to be not translatable — for example an image input of some component
6. 🟢 debt: currently, XB fields can only exist on XB's own Page content entity type (base field definition) and on the article bundle of Node entities (configurable field) — expanding that is out of scope here and is in scope for #3498525: [META] Allow Canvas to be used on any content entity type (bundle), as long as it has certain entity type characteristics. This poses no additional risks for XB's data storage, because this already covers the gamut of possible kinds of XB fields on content entities.
7. 🟢 feature: full read/write support of XB trees in JSON:API (#3521002: [META] Maintainable client-side data model + internal HTTP API's #3499632: Remove `ClientServerConversionTrait` (C→S) and `ComponentTreeItemList::getClientSideRepresentation()` (S→C) in favor of (de)normalizers), including the ability to filter (→ #3523844: Spike: Explore adding configuration options to the tree item formatter to support alternate use-cases).
8. 🟢 feature: ability to store variants of component trees for use cases like personalization (i.e. different component subtree for Belgians & Brits vs everyone else → #3525564: [META] Experience Builder Personalization), responsive design (i.e. breakpoint-specific overrides), and arbitrary future use cases (→ #3525746-6: Update the React client preview view)

Beta blockers (July 2025)

Beta blockers (tag: beta blocker) MUST happen before #3515932: Milestone 1.0.0-beta1: Enable creation of non-throwaway sites.

⚠️ As part of this, we're aiming to ideally close #3477428: [PP-1] Refactor (or decide not to) the XB field type to be multi-valued, to de-jsonify the tree, and to reference the field_union type of the prop values and #3440578: [PP-2] JSON-based data storage proposal for component-based page building, but both of those cover many aspects. They're being tackled piece-by-piece by the beta-blocking issues below, as originally proposed by @larowlan at #3477428-44: [PP-1] Refactor (or decide not to) the XB field type to be multi-valued, to de-jsonify the tree, and to reference the field_union type of the prop values.

(Numbered lists must happen one after the other, bulleted lists can happen in parallel.)

✅ Dependencies/usage beta blockers

Precise dependency information for every XB component tree: every config entity with a component tree AND every revision of an XB field instance in content entities, because that allows strong guarantees, and is essential for explicit input schema evolvability

1. ✅ #3457504: XB field type: calculate all dependencies, store them, surface in new Component "Audit" operation
2. ✅ #3521202: Store XB field type's "deps_*" columns in separate table to allow efficient querying
3. ✅ #3460230: Component entity config dependencies are incomplete: missing ComponentSourceInterface::calculateSettingsDependencies() to compute deps for field storage + instance settings + widget
4. ✅ #3544604: Calculating dependencies of `ReferenceField(Type)PropExpression` is missing intermediate dependencies

✅ Field type storage (content entities) schema beta blockers

Given lessons in the ecosystem over the past decade, plus what we know about functionality we still intend to add to XB, organize the data storage to avoid future backwards compatibility breaks, and allow data storage for new features to be easily added later.


1. ✅ Ensuring multi-value (type: array) explicit inputs (e.g. list of integers for sparkline component, list of images for image gallery component, etc.) can be supported later: #3467870: Support `{type: array, …}` prop shapes
2. ✅ 1 row/revision with 2 enormous JSON blobs → 1 row/component instance/revision with a single tiny JSON blob: #3468272: Store the ComponentTreeStructure field property one row per component instance
3. ✅ content templates: XB field on content entities should be able to store >1 component tree: one per exposed slot in the governing ContentTemplate, if any #3519352: Content templates, part 3b: store exposed slot subtrees on individual entities
4. ✅ New name field property for additional functionality: #3460958: Data storage support for allowing Content Creator to name component instances for the specific context
5. ✅ versioned Components + associated storable prop shapes (field types + settings) (blocker for #3524751: [later phase] Component Source plugins: generalized support for schema changes of explicit inputs): #3523841: Versioned Component config entities (SDC, JS: prop_field_definitions, block: default_setting, all: slots for fallback) + component instances refer to versions ⇒ less data to store per XB field row
6. ✅ bike shed finalize the field type's field property names: #3528167: Rename `version` field property to `component_version`
7. ✅ ensure this architecture allows for efficient updating of existing component trees when explicit input schemas (block settings, SDC props …) change: #3501708: Prove that it *will* be possible to apply block settings update paths (assuming #3521221 in core) to stored XB component trees in config/content
8. ✅ #3528528: `ComponentInputs::getPropSources()` needs to take `GeneratedFieldExplicitInputUxComponentSourceBase::rawInputValueToPropSourceArray()` into account
9. ✅ #3528499: Revisit storage of dependencies in separate table now we have separate deltas per component + remove plugin dependencies + make component version usages auditable
10. ✅ #3520923: [PP-1] Use `json` schema type for SQLite and remove `text` workaround

✅ Configuration storage schema beta blockers

• ✅ #3502982: Rename ComponentSource settings' `plugin_id` to `local_source_id` or similar to not bias towards source plugins that don't use plugins under the hood
• ✅ #3521137: XB's storing JSON blobs should not be visible in exported config nor to test authors
• ✅ ensure the versions are deterministic hashes: #3528159: Ensure deterministic version hashes for ComponentSource-specific settings, thanks to config schema-powered normalization
• ✅ ensure the deterministic hashes capture all crucial aspects: #3528362: Deterministic Component version hash should depend not only on source-specific settings, but also slots + explicit input schema
• ✅ #3526127: Ensure deterministic config export order of config-defined component trees

✅ Validation beta blockers

• ✅ #3473289: XB Component config entity's `status` may only be `true` if it meets all of XB's requirements
• ✅ #3526818: Tighten validation of `parent_uuid` and `slot` on XB fields to match the strictness of config
• ✅ #3519891: Constrain slot names allowed by XB in Components (and in its component tree)

Stable blockers (October 2025)

Stable blockers (tag: stable blocker) do not block #3515932: Milestone 1.0.0-beta1: Enable creation of non-throwaway sites, but would be nice-to-have at that point in time. They MUST happen before #3517885: Milestone 1.0.0: Production Sites.

Field type storage (content entities) schema stable blockers

• ✅ #3528284: Add e2e tests that prove we can edit an old version of a component

Configuration storage schema stable blockers

• ✅ #3533458: Change CodeComponentDataProvider::getRequiredXbDataLibraries() to base its logic on information provided by the front-end rather than on naive string/regex matching
• ✅ #3538537: Update path tests should also run against non-SQLite DBs
• ✅ #3548165: Components in `ContentTemplate`s and `PageRegion`s are not always rendered in correct order
• #3538273: Use AST to identify resources fetched by CodeComponents so they can be preloaded/prefetched
• #3460232: [PP-1] Support XB config entities with component trees that reference File/Media, use target_uuid and embed — aka: self-contained content entity dependencies
• #3526706: Stop assuming default Field Widget settings suffice — add Field Widget settings support to `experience_builder.generated_field_explicit_input_ux: prop_field_definitions`
• #3503272: JavaScriptComponent config entities should have mutable machineNames Assigned to: balintbrews
• #3464042: Add ComponentAuditabilityTest

Validation stable blockers

• ✅ #3471026: Harden UUID validation in ComponentTreeStructureConstraintValidator
• ✅ #3525759: SdcPropKeysConstraintValidator::validate() should complain about extraneous keys too, not just missing keys
• ✅ #3526721: Require Drupal 11.1.8 and remove `type: field.value.language` work-around
• ✅ #3526707: Tighten validation of `experience_builder.generated_field_explicit_input_ux: prop_field_definitions.[%key].default_value`
• ✅ #3538503: Disallow component trees with `component_version: active`
• ✅ #3526716: Tighten validation of `experience_builder.generated_field_explicit_input_ux: prop_field_definitions.[%key].expression`
• ✅ #3538487: Don't allow passing uncollapsed inputs if using default expression

Security stable blockers

• ✅ #3546996: Component instances populated by linked `DynamicPropSource`s, but inaccessible entity/field: fall back to `NULL` for these values, triggering fallback rendering
• ✅ #3554184: Bubble cacheability of resolved props values and access results + `PropSourceBase::evaluate()` does not return cacheability at all

Other stable blockers

• ✅ #3528723: Component::onDependencyRemoval() should be an uninstall validator if the dependency being removed is a module or theme
• ✅ #3537783: ComponentTreeItem must override `::mainPropertyName()` because it has no `value` field property

Post-stable priorities (after October 2025)

Post-stable priorities (tag: post-stable priority) do not block any release, and are things we'd like to see happen, but are intentionally NOT happening any time soon. We're highly confident that these will be net additions without the need for backwards compatibility breaks.

• Field type storage (content entities) schema full feature scope blockers
  • #3524406: [later phase] [Needs design] [PP-1] Provide API for finding and UI for surfacing dangling/dead component subtrees — aka garbage clean-up Assigned to: lauriii
  • #3498525: [META] Allow Canvas to be used on any content entity type (bundle), as long as it has certain entity type characteristics
  • #3526189: Spike: explore merits of one Canvas field per exposed slot in content templates
  • #3528458: Component instances in exposed `ContentTemplate` slots must use the exposed slot's machine name
• To be categorized:
  • 🟢#3518272: Support all entity types with configurable displays for content templates — note that the stable-blocking subset already landed — see #3518272-7: Support all entity types with configurable displays for content templates for details.
  • #3526703: Adopt `AtLeastOneOf` validation constraint for cardinality validation
  • #3523844: Spike: Explore adding configuration options to the tree item formatter to support alternate use-cases Assigned to: larowlan
  • #3522953: [later phase] Make Component `audit` operation performant/scalable
  • #3530051: Update `ComponentAudit::getConfigEntityDependenciesUsingComponent()` to support component versions
  • #3532712: Audit use of ::loadUnchanged and any routes that might need to load the latest revision
• clean-up:
  • #3530097: [later phase] [PP-1] Prune unused `Component` config entity versions
  • #3495625: Remove ComponentTreeItemList::ROOT_UUID from hydration and client-to-server conversion
  • #3518696: [2.x] [11.3-only] Remove `BetterConfigExistsContraint` and move back to `ConfigExistsContraint`
  • #3481188: [later phase] Gracefully handle deleted regions in PageTemplate config entities"
• asymmetric translation explicit input locking: across al component instances and/or per component instance
  • Add a untranslatable_inputs field property, to allow a content author to lock an explicit input of a single component instance — e.g. a particular image component instance may have contain an image that should be the same across all translations (maybe it is a photo of the CEO), but another image component instance may have an image that should be translatable (e.g. locale-specific names for the same product: "Lotus" speculoos cookies in Belgium, "Biscoff" in the US)
    

    diff --git a/config/schema/experience_builder.schema.yml b/config/schema/experience_builder.schema.yml
    index 900a0b96e..ea54e8b34 100644
    --- a/config/schema/experience_builder.schema.yml
    +++ b/config/schema/experience_builder.schema.yml
    @@ -264,6 +264,10 @@ experience_builder.component_tree_node:
         inputs:
           type: ignore
           label: 'Input values for each component in the component tree'
    +      translatable: true
    +    untranslatable_inputs:
    +      type: ignore
    +      label: 'Untranslatable input values for each component in the component tree'
     
     # Based on core/assets/schemas/v1/metadata-full.schema.json#$defs.slotDefinition
     # @todo Tighten & add validation in https://www.drupal.org/project/experience_builder/issues/3519891
    diff --git a/src/Plugin/Field/FieldType/ComponentTreeItem.php b/src/Plugin/Field/FieldType/ComponentTreeItem.php
    index 6fc12e593..c9a95873b 100644
    --- a/src/Plugin/Field/FieldType/ComponentTreeItem.php
    +++ b/src/Plugin/Field/FieldType/ComponentTreeItem.php
    @@ -96,6 +96,10 @@ use Symfony\Component\Validator\ConstraintViolationList;
             'component_id',
           ],
         ],
    +    'untranslatable_inputs' => [
    +      'label' => new TranslatableMarkup('Untranslatable component input values'),
    +      'translatable' => FALSE,
    +    ],
       ],
     )]
     class ComponentTreeItem extends FieldItemBase {
    @@ -257,7 +261,16 @@ class ComponentTreeItem extends FieldItemBase {
               'not null' => TRUE,
             ],
             'inputs' => [
    -          'description' => 'The input for this component instance in the component tree.',
    +          'description' => 'The inputs for this component instance in the component tree.',
    +          'type' => 'json',
    +          'pgsql_type' => 'jsonb',
    +          'mysql_type' => 'json',
    +          // @todo Change back to 'json' once https://www.drupal.org/i/3487533 is resolved.
    +          'sqlite_type' => 'text',
    +          'not null' => FALSE,
    +        ],
    +        'untranslatable_inputs' => [
    +          'description' => 'The untranslatable inputs for this component instance in the component tree.',
               'type' => 'json',
               'pgsql_type' => 'jsonb',
               'mysql_type' => 'json',
    @@ -333,6 +346,10 @@ class ComponentTreeItem extends FieldItemBase {
           ->setLabel(new TranslatableMarkup('Input values for each component in the component tree'))
           ->setRequired(TRUE);
     
    +    $properties['untranslatable_inputs'] = DataDefinition::create('component_inputs')
    +      ->setLabel(new TranslatableMarkup('Input values for each component in the component tree'))
    +      ->setRequired(FALSE);
    +
         return $properties;
       }
    

  • Related, add a new asymmetrically_translated_explicit_inputs setting to type: experience_builder.component_source_settings.* to allow site builders to lock an explicit input of ALL instances of a component:
    

    diff --git a/config/schema/experience_builder.schema.yml b/config/schema/experience_builder.schema.yml
    index 19722e2bc..bafe44264 100644
    --- a/config/schema/experience_builder.schema.yml
    +++ b/config/schema/experience_builder.schema.yml
    @@ -266,6 +266,12 @@ experience_builder.component_source_settings.*:
           type: string
           label: 'The intra-source ID of this component in this source'
           constraints: {}
    +    asymmetrically_translated_explicit_inputs:
    +      type: sequence
    +      label: 'The list of explicit inputs of this component that should only be stored in the default translation'
    +      sequence:
    +        # @todo Validate: must match a concrete explicit input
    +        type: string
    

• storage efficiency — see #13 through #15 on #3523842 for the most current overview (mid-May 2025). This is about deduplication of blobs of the per-component instance JSON blobs that contain the explicit inputs for that component instance. Doing both of these is best, but doing either one will diminish the impact of the other.
  • core: #3469082: Add way to "intern" large field item values to reduce database size by 10x to 100x for sites with many entity revisions and/or languages
  • core: #2770417: Revision garbage collection and/or compression
  • XB — probably we won't do this after 1 or both of the above land: #3523842: Spike: Explore storing a hash lookup of the ComponentInputs field property: one hash per component instance
• #3463996: [META] When the field type, storage/instance settings, widget, expression or requiredness for an SDC/code component prop changes, the Content Creator must be able to upgrade Assigned to: penyaskito
• #3524345: Avoid deleting `PageRegion`s & `Pattern`s when uninstalling field type-providing module: implement `::onDependencyRemoval()` , replace with default StaticPropSource