[META] Production-ready ComponentSource plugins

Discussed #explicit-input-schema-change-strategies in detail with @longwave just now. Here are the meeting notes:

1. @longwave came up with a really really interesting proposal in #3501708-9: Prove that it *will* be possible to apply block settings update paths (assuming #3521221 in core) to stored XB component trees in config/content for allowing us to reuse all of the existing block update path logic, seemingly without the need for additional logic/burden on block plugin maintainers!
   However … it can't actually work because we won't know which of those post-update hooks are associated with which block plugins! In theory, we could run all those block-targeting post-update hooks, but then we'd need to track for every single block-sourced component instance which update hooks have already been executed (equivalent to how core does the system-wide UpdateRegistry). That doesn't scale, just like it doesn't scale to update potentially millions of (revision) rows.
2. We agreed that different update strategies (perhaps the 3 in ExplicitInputSchemaChangeStrategyEnum in the current IS) will be necessary. And there may be a need for even more. It really depends on how a particular component source functions. Future ones: layout types, paragraph types, web components, external API shenanigans, etc. — it's literally anything. So we cannot lock ourselves into one corner.
3. … but short-term, the primary challenge is block plugins. (Worst case: we disable the SDCs and code components which add new required properties and let #3517941: [META] Robust component instance error handling during hydration+rendering handle it for us until that SDC/code component is edited and gets the missing props.)
4. The "track different explicit input schema versions" part described in the sibling meta needs massive rethinking because "provider version" is meaningless in a world of post-update hooks (which is the block plugin reality today!)

Conclusion: both my ideas (articulated in this issue + #3520449) and @longwave's (at #3501708-9) have been invalidated 😅 But we both agree that #3457504: XB field type: calculate all dependencies, store them, surface in new Component "Audit" operation is an important step towards more clarity on how we might solve this.

#3518832: ComponentSource robustness: add `ComponentSourceTestBase::testGetClientSideInfo()` and #3521072: Auto-saved Javascript Components CSS changes do not work with CSS aggregation landed 🚢

Replacing #3497990 with #3518250: [PP-1] Handle components instances rendering nothing: wrap them in a container per @lauriii at #3497990-18: [later phase] Support Block's PreviewAwarePluginInterface (and similar for other component sources).

Added #3522217: JsComponent-sourced rendered component instances lack cacheability of underlying config entities (+ first-party imports + auto-save tag when previewing).

#3520617: BlockComponent::checkRequirements() should disallow any block plugins whose definition declares it requires certain contexts and #3502982: Rename ComponentSource settings' `plugin_id` to `local_source_id` or similar to not bias towards source plugins that don't use plugins under the hood landed.

Added:

1. #3503038: Enable candidate `DynamicPropSource` suggestions for code components: refactor `GeneratedFieldExplicitInputUxComponentSourceBase` and `FieldForComponentSuggester` to need only SDC's ComponentMetadata, not SDC plugin instances
2. #3522673: Previews of SDCs with default image props broken

Added #3521843: JS component slots and props don't appear in the preview canvas until published.

#3522673: Previews of SDCs with default image props broken landed a while ago. #3508725: ComponentMetadataRequirementsChecker::check() should validate that the example(s) actually match the JSON schema landed just now.

#3520944: Make menu blocks (block.settings.system_menu_block:*) fully validatable landed in core.

Added #3499352: SDCs should only have get HTML comments injected when `renderComponent(isPreview: TRUE)` and #3473761: Render a placeholder DIV inside empty Regions/Slots in the preview.

Added #3513589: [upstream] Decouple GeneratedFieldExplicitInputUxComponentSourceBase from SDC plugins, #3500795: [PP-1] Implement client-side validation of block settings, #3523496: Block component instance form values not processed by validation/submit handlers.

#3508937: Global AssetLibrary should render with its auto-saved state (if any) when rendered in the XB UI and #3518722: Test coverage to prove that BlockPluginInterface cacheability is correct when rendered via XB landed.

New category in the issue summary: Source's explicit inputs:. Prompted by the addition of #3521444: Required image field becomes non-functional after publishing with empty field.

Also marked #3492358: Provide ComponentSource-appropriate and -specific APIs to allow component instances to behave differently while being previewed as less urgent.

#3523496 was in there twice 🙈

👉️🔄

Moved #3503087: Disallow SDCs with `type: string, enum: …` with an empty string listed to convey optionality: the prop should then just be optional to "discovery" thanks to superb feedback from somebody in the community I've not seen before — wonderful way to start the day :)

Also: #3521128: Ensure XB works with CSS + JS aggregation, including for (auto-saved) code components landed!

Landed:

1. #3519168: Handle components provided by ComponentSources EXPLICITLY disappearing — enables deleting JS components that are in use
2. #3522217: JsComponent-sourced rendered component instances lack cacheability of underlying config entities (+ first-party imports + auto-save tag when previewing)
3. #3499352: SDCs should only have get HTML comments injected when `renderComponent(isPreview: TRUE)`
4. #3484682: Handle update and delete of Block `Component`s, plus react to dependency removal

Added: #3525374: Include compiled CSS from global asset library when displaying component library preview thumbnails.

Updated the issue summary to more clearly distinguish between:

1. beta blockers (i.e. necessary for #3515932: Milestone 1.0.0-beta1: Enable creation of non-throwaway sites)
2. stable blockers (i.e. necessary for #3517885: Milestone 1.0.0: Production Sites)

#3525374: Include compiled CSS from global asset library when displaying component library preview thumbnails landed.

Removed #3460230: Component entity config dependencies are incomplete: missing ComponentSourceInterface::calculateSettingsDependencies() to compute deps for field storage + instance settings + widget, because that belongs in #3520449: [META] Production-ready data storage, where it is already being tracked.

Moved #3518250: [PP-1] Handle components instances rendering nothing: wrap them in a container and #3473761: Render a placeholder DIV inside empty Regions/Slots in the preview to stable blocker (but in order to understand whether that's appropriate, I actually … kinda ended up completely fixing that last one 😅)

Triaged the list of Component source issues tagged stable blocker. Added the following missing stable blockers:

1. #3513590: `ComponentSourceInterface::inputToClientModel()` needs to support passing a host entity
2. #3492366: Display SDC prop `description` as the form element description in component instances form
3. #3497990: [later phase] Support Block's PreviewAwarePluginInterface (and similar for other component sources)

And also one overlooked beta blocker: #3516602: SDC `enum` props should have human-readable labels: use `meta:enum`

Now that triaging is done, I realized/discovered that only 3 known beta blockers remain:

• sdc,js: #3516602: SDC `enum` props should have human-readable labels: use `meta:enum`
• sdc#3523625: ComponentMetadataRequirementsChecker::check() should validate no prop is of `type: object` without a proper `$ref`
• sdc,js: #3503087: Disallow SDCs with `type: string, enum: …` with an empty string listed to convey optionality: the prop should then just be optional
• sdc: #3522164: Handle update and delete of SingleDirectoryComponent `Component`s, plus missing config dependencies — AFAICT fixed by a larger infra issue, being confirmed.)

Added #3525691: Rename ComponentSourceInterface's singular method names to plural as a stable blocker.

#3522164: Handle update and delete of SingleDirectoryComponent `Component`s, plus missing config dependencies was confirmed as a duplicate 👍

Fix HTML.

Some crucial text was lost in the #47 reshuffle.

#3523625: ComponentMetadataRequirementsChecker::check() should validate no prop is of `type: object` without a proper `$ref` is in.

Added #3514672: [Needs design] DX & authoring experience: support `default` in addition to `examples[0]`, enables updating a component input across all existing instances to a new Post-stable targets section.

Added these stable blockers:

• block: #3524399: `BlockComponent::validateComponentInput()` allows garbage to pile up
• sdc,js: #3524401: `GeneratedFieldExplicitInputUxComponentSourceBase::validateComponentInput()` allows garbage to pile up

Added #3509265: Auto-saving of blocks needs to handle string-to-bool fixing.

Added #3504925: Preview is not accurate for the site branding block: missing Olivero's CSS variables.

#3503087: Disallow SDCs with `type: string, enum: …` with an empty string listed to convey optionality: the prop should then just be optional is in!

Added #3526967: Disabled SDC components are re-enabled after cache rebuild as a new beta blocker.

#3526967: Disabled SDC components are re-enabled after cache rebuild landed.

Down to one remaining beta blocker again: #3516602: SDC `enum` props should have human-readable labels: use `meta:enum`.

Added @phenaproxima's #3526045: Settle on final name for `ComponentSource` plugins (`ComponentType`?) + expand their scope (discovery, maybe more?).

The beta parts are 99% done 🥳 — the only remaining one is #3516602: SDC `enum` props should have human-readable labels: use `meta:enum`, which is in the final stages, and the "land upstream support for this" part is already complete: #3493070: SDC `enum` props should have translatable labels: use `meta:enum`. 😊

h3 → h2

Moved #3502371: Make "Page title block" work A) also outside regions, B) on the only routes XB currently supports: content entity routes here from #3521002: [META] Maintainable client-side data model + internal HTTP API.

New stable blocker: #3526118: CSS assuming block-plugin-rendered-in-a-wrapper-for-the-block-config-entity does not apply to XB: either A) stay the course, B) simulate the same markup structure.

Given #3526644-9: Make basic webform experience work with Drupal Canvas, I think we actually need something like the all-props SDC but for block plugins.

I suspect we actually need to expand what \Drupal\experience_builder\Plugin\ExperienceBuilder\ComponentSource\BlockComponent::checkRequirements() does, because block forms can use arbitrary render arrays, and those are definitely not guaranteed to just work in XB.

So I think BlockComponent::checkRequirements() would at minimum need to be expanded with calling ::blockForm() and traversing it to see if only "known to be working" #types are present 😅

It seems that in core only the following #types are used:

• actions
• checkbox
• container
• details
• entity_autocomplete
• fieldset
• select
• submit
• textfield

Some of those likely will require some work in XB's Semi-coupled theme engine AFAICT.

Crediting @bnjmnm for putting #3526644 on my radar!

Together with @bnjmnm I articulated a proposal at #3500795-8: [PP-1] Implement client-side validation of block settings 😊

Added #3532414: Follow-up for #3500386: tighten `::collapse()` to improve data integrity, because new props added to auto-saved code components cannot have a widget anyway as a stable blocker.

#3473761: Render a placeholder DIV inside empty Regions/Slots in the preview and #3518325: [11.2-only] BlockComponent::checkRequirements() requires block settings to be fully validatable, but XB pretended for demo purposes Views blocks are … now: error! landed!

Added #3532514: Gracefully handle components in active development: ensure great DX.

Added #3540082: JavaScriptComponent::getComponentUrl() gets calculated twice on render for each component.

ℹ️ @lauriii (together with @effulgentsia and I) decided to descope a stable ComponentSource plugin API to after 1.0.

It'll still be around for contrib to experiment, but we realistically won't have the time to actually make it a stable PHP API with a BC promise before DrupalCon Vienna.

• Added #3541125: Integer BlockComponent inputs (i.e. block settings) are incorrectly saved as strings.
• Closed #3484669: Define how block settings should be presented in the UI.
• Landed #3523496: Block component instance form values not processed by validation/submit handlers, which allowed closing #3509265: Auto-saving of blocks needs to handle string-to-bool fixing.

#76 has been reflected over at #3454519-52: [META] Support component types other than SDC, block, and code components too now.

Added

• #3546208: [Needs design] Updating the `group` key in SDC's *.component.yml does not update it in Canvas editor (even after cache clear)
• #3546209: Support `replaces`: 2 SDCs with one replacing the other — Canvas creates `Component`s for both

Added #3546903: Disallow using any ComponentSource other than SDC/JS/Block unless `canvas_dev_mode` is installed, which already landed thanks to swift @lauriii approval!

New: #3544756: ComponentInstanceForm displays default value in the component form when value is empty. — and landed overnight.

Added #3549793: ComponentInstanceForm fails to load for block components without any config.

Added #3502376: Adding an empty block breaks annotation rendering and #3542194: 500 error when using Test Block Components.

AFAICT we've got our first example of a slot-only ComponentSource: the Layout source, which will enable a Layout Builder-to-Canvas migration.

Added #3542102: Prevent publishing when `block` component instance values do not #validate.

New:

• #3565718: Validate `GeneratedFieldExplicitInputUxComponentSourceBase`-powered Component config entity props' `required` value matches the actual source for the `active` version
• #3565755: Changing a slot's metadata (title/examples) should not generate a new Component config entity version

New: #3550882: Empty (optional) date prop throws error upon reload, due to bug in client-side transform, #3530808: Example values for SDC props powered by datetime fields (in core, more in contrib) are not config schema-conformant because Canvas does not call `FieldItemListInterface::processDefaultValue()`.

#3550882: Empty (optional) date prop throws error upon reload, due to bug in client-side transform and #3524401: `GeneratedFieldExplicitInputUxComponentSourceBase::validateComponentInput()` allows garbage to pile up landed!

Added #3572850: "block" ComponentSource plugin should never store inputs for `label` and `label_display`, because they're not available for Canvas users.

#3574120: Validate props for code components when rendering, like SDCs do was created yesterday and landed today :)

New: #3576578: Inform SDC/code component developers of nonsensical `type: array` prop restrictions: `maxItems <2` and #3576805: DX: reorganize test components: `canvas_test_sdc` should have its (intentionally invalid) SDCs moved to `canvas_test_sdc_invalid`.

New: #3578142: [upstream] New Views blocks are not available to be placed until cache clear.

New: #3558721: Regression: can no longer access previously stored values for block component instances.