[META] Untie config validation from form validation — enables validatable Recipes, decoupled admin UIs …

This can't be done during 8.0.0 so postponing to 8.1.x

@alexpott, @cilefen, and I discussed these issues today during a CMI meeting.

The title of this issue is a little misleading as "config validation" is a different thing from the kind of validation for data done on form submission. I was going to retitle it to "use API validation for configuration entity data" or something like that. Seems like implementation-wise there is overlap with #1928868: Typed config incorrectly implements Typed Data interfaces and the potential blocker #1818574: Support config entities in typed data EntityAdapter?

Does this just mean perform config schema validation every time config is saved?

And I don't think this needs to be postponed anymore? This can be worked on now.

Well, conceptually #1928868: Typed config incorrectly implements Typed Data interfaces would solve the underlying API capabilities. This issue is then more about organizing an effort to add all the various constraints .

I reported over at #2952037-10: [meta] Add constraints to all simple configuration that as of #2969065: Use typed config validation constraints for validation of cdn.settings simple config (which is using #1928868: Typed config incorrectly implements Typed Data interfaces which @dawehner pointed to in #9), the https://www.drupal.org/project/cdn module that I maintain uses config validation. As part of that, it is now relying completely on config validation to provide form validation too: https://git.drupalcode.org/project/cdn/blob/8.x-3.x/cdn_ui/src/Form/Vali... — this is working great 🥳.

I think that in Drupal 9 we could deprecate ConfigFormBase and require everything to move to ValidatableConfigFormBase by Drupal 10?

Initial patch attached.

I think I'd prefer a flag on the ConfigFormBase that defaults to the current behaviour but you can opt in to the new behaviour. And then trigger a deprecation error if it is not set to the new behaviour.

Works for me :)

Forgot to say - really lovely to see some progress on this @Wim Leers++

After a nudge from larowlan in #australia-nz slack - suggest we remove the meta from this ticket.
Many of the blocking issues mentioned are resolved now, it's ready for implementation, there's even a patch.

For the 34 current followers: good news, we have significant progress on this front despite the silence in this issue! 🤓🥳

Simple config

#2952037: [meta] Add constraints to all simple configuration would be nice but it would only solve validation for simple config. The CDN contrib module has been using that for >3 years (see #2969065: Use typed config validation constraints for validation of cdn.settings simple config).

Config entities

The real challenge is config entities. Well … the CKEditor 5 module (experimental module in Drupal core since 9.3) is about to become stable and has no validation logic in its ::validateConfigurationForm() implementation — it's all in validation constraints which are applied using the constraints key in the config schema! 🥳

Applying it to all config entities in Drupal core

So … the next step here is to make it easy to repeat this pattern, and to mass-convert existing form validation logic to validation constraints. #3231342: [PP-2] Introduce ConfigEntityForm to standardize use of validation constraints for config entities will introduce the infrastructure to make that easy.

It'll actually be relatively simple ^(mostly)

The CKEditor 5 validation case is more complex for three reasons

1. CKEditor 5 configuration is stored in editor config entities, which are tightly coupled and must be kept in sync with filter_format config entities — #3231354: [PP-1] [META] Discuss: merge the Editor config entity into the FilterFormat config entity would remove that complexity
2. CKEditor 5 is merely one Text Editor plugin, a different editor (say, TinyMCE) would of course need different validation constraints. Many config entities do not use plugins, making things a lot simpler.
3. The CKEditor 5 Text Editor plugin itself has a plugin manager, allowing to add plugins to this plugin 😅, making these config entities even more complex to validate.

What's next? What does this bring us?

The above is why I'm optimistic about expanding this to other config entities. Let's make a big push forward in Drupal 10.1? 🤓

This is what made the decoupled Admin UI initiative nearly impossible. Today, we have the proven blueprints for making #2300677: JSON:API POST/PATCH support for fully validatable config entities possible!

But that's not the only advantage! Examples:

• It'll become possible to be much more confident about deployments.
• We would not depend anymore on implicitly secure configuration (because a form only allows certain values to be submitted), but would be able to be confident that no unexpected values exist in configuration that would otherwise be able to trigger edge cases in the code using that configuration.
• Contrib/custom modules modifying config entities (upon installation or at runtime) could be prevented from being able to incorrectly modify configuration anymore.
• …

Maybe we can start with Vocabulary config entities, like @fago proposed ~8.5 years ago in #2002174: Allow vocabularies to be validated via the API, not just during form submissions? 🤓

Big news!

Huge tangible progress in #3324150: Add validation constraints to config_entity.dependencies by @phenaproxima (which brings some validation to ALL config entities! 🤩), and I also got started on #3324140: Convert field_storage_config and field_config's form validation logic to validation constraints, both will be used by https://www.drupal.org/project/field_ui_modern. 🤓

Also working to expose the schema information through JSON:API Schema, and working on bugfixes there: #3324769: Schema incorrect for config entity types with multi-part IDs (field_storage_config, field_config, entity_view_mode, entity_form_mode, entity_view_display and entity_form_display) + #3324824: Schema incorrect for config entity "fields" that are Maps and Sequences.

Update: created #3324982: [Proposal/Exploration] Expose generating JSON Schema that matches the config schema for decoupled admin UIs based on trying to expose the pre-existing config schema for field_storage_config via JSON:API Schema. In there, I proposed an exploration direction to find out how we can build on top of the existing JSON Schema ecosystem and the JSON Schema Validation RFC.

Key things we should evaluate (and which this meta issue is tangentially related to):

• Is JSON Schema is expressive enough for us to generate UIs from (of course, customizing/optimizing auto-generated UI is still something that should always remain available)?
• Is JSON Schema is expressive enough for us to be able to map config schema onto it?
• … and related: does adding validation constraints to config schema (which is what this issue is about) move us further away from being able to use JSON Schema 🤔 … or … should we constrain our constraints (pun intended 🫣) to maximize declarativeness to allow client-side validation, but … then where to draw the line, because in the end it can really only be the server to do the final validation, not the client, even if only for race conditions/collisions (e.g. 2 users choosing the same ID)

Now that #3324150: Add validation constraints to config_entity.dependencies is RTBC and #3324140: Convert field_storage_config and field_config's form validation logic to validation constraints is moving forward, I created #3324984: Create test that reports % of config entity types (and config schema types) that is validatable to help us track overall progress! 😊

The very first piece of system-wide config entity validation landed! 🤩🚀 See #3324150: Add validation constraints to config_entity.dependencies. Next up: #3324140: Convert field_storage_config and field_config's form validation logic to validation constraints and #3324984: Create test that reports % of config entity types (and config schema types) that is validatable.

Huge progress!

1. #3324984: Create test that reports % of config entity types (and config schema types) that is validatable is working, and currently reports:
   

   Config types

     ℹ️   4.62% validatable (29 of 628 config types — excludes base types)
     ℹ️  25.94% average config type validatability
   …
     🔴   0% block.settings.node_syndicate_block
     🔴   0% block.settings.search_form_block
     🔴   0% block.settings.statistics_popular_block
     🔵  75% block.settings.system_branding_block
     🟡  25% block.settings.system_menu_block:*
     🔴   0% block.settings.views_block:*
     🔴   0% block.settings.views_exposed_filter_block:*
     🔴   0% block_content.type.*
     🟡  30% block_settings
     🟡  43% book.settings
     🟢 100% ckeditor5.element
     🟢 100% ckeditor5.plugin.ckeditor5_alignment
     🟢 100% ckeditor5.plugin.ckeditor5_heading
     🟢 100% ckeditor5.plugin.ckeditor5_imageResize
     🟢 100% ckeditor5.plugin.ckeditor5_language
     🟢 100% ckeditor5.plugin.ckeditor5_list
     🟢 100% ckeditor5.plugin.ckeditor5_sourceEditing
     🔵  80% ckeditor5.plugin.ckeditor5_style
     🟢 100% ckeditor5.plugin.media_media
     🟢 100% ckeditor5.toolbar_item
     🟡  42% ckeditor5_valid_pair__format_and_editor
     🔴   0% color_hex
     🟡  50% comment.settings
   …
     ℹ️ 36.68% validatable property paths (1369 of 3732 property paths — this excludes property paths for base types)
   …
   

   Config entity types

     ℹ️   0.00% validatable (0 of 30 config types — excludes base types)
     ℹ️  24.05% average config type validatability
   …
     🔵  78% editor.editor.*
   …
     ℹ️ 35.40% validatable property paths (194 of 548 property paths — this excludes property paths for base types)
   …
   

2. That report-generating test 👆 shows what a big impact #3324150: Add validation constraints to config_entity.dependencies has had; it has ensured that all config entity types have at least a part that is already validatable!
3. #3324150: Add validation constraints to config_entity.dependencies is having a strong secondary impact too: its ValidKeys: ['<infer>'] is now being applied to all type: mapping config, and hence helps ensure a huge part of configuration is already partially validatable! (#3324984: Create test that reports % of config entity types (and config schema types) that is validatable is applying that generically.)
4. #3324140-8: Convert field_storage_config and field_config's form validation logic to validation constraints shows very clearly how we can use #3324984 to our advantage to

So … now that we've arrived at this point … it's finally time to articulate a plan to crowdsource updating all of Drupal core. Added that to the issue summary! 🚀

HTML fix.

Typo fix.

Yay, the plan from #34 is starting to come to fruition: #2920682: Add config validation for plugin IDs landed with a new validation constraint and used it in two places (somewhere in Block config entity's schema and somewhere in Text Editor config entity's).

Results of the output from #3324984: Create test that reports % of config entity types (and config schema types) that is validatable's test coverage for drupal/core at the time of #34 and today are attached and diffed.

As expected:

• ℹ️ 24.05% → 24.20% average config entity type validatability
• 🟡 27% → 29% block.block.*
• 🔵 78% → 80% editor.editor.*
• ℹ️ 35.40% → 35.77% validatable property paths (194 → 196 of 548 property pathss)

and

• ℹ️ 4.62% validatable (29 of 628 config types — excludes base types) ←←←←← unchanged! 👍
• ℹ️ 25.94% → 25.95% average config type validatability
• ℹ️ 36.68% → 36.74% validatable property paths (1369 → 1371 of 3732 property paths — this excludes property paths for base types)

And so it begins … 🤓🤞

FYI: The next step forward is happening at #2920678-113: Add config validation for the allowed characters of machine names! 🥳

For the ~50 interested people: @phenaproxima proposed to work on #3341682: New config schema data type: `required_label` next. I'm working on #3324140: Convert field_storage_config and field_config's form validation logic to validation constraints 👍

#2920678: Add config validation for the allowed characters of machine names landed!

#3373570: #3360991 changed TypedConfigManager::createFromNameAndData() in 10.1.x added 10.1 support to the Config Inspector module.

Focus now shifts to:

1. Drupal-core-wide infrastructure: #3361534: KernelTestBase::$strictConfigSchema = TRUE and BrowserTestBase::$strictConfigSchema = TRUE do not actually strictly validate needs review and is committable, with 4 follow-ups to stop ignoring some validation errors and make issues manageable in scope
2. Ecosystem enabler: ValidatableConfigFormBase: #3364506: Add optional validation constraint support to ConfigFormBase
3. Expand validatability: type: label validation: #3341682: New config schema data type: `required_label`
4. Bug/weakness in existing config schema infrastructure: #3364109: Configuration schema & required values: add test coverage for `nullable: true` validation support
5. Bug/weakness in existing config schema infrastructure: #3364108: Configuration schema & required keys (blocked by #3364109)
6. First config entities to fully use config validation (last but not least!) continue chipping away at #3324140: Convert field_storage_config and field_config's form validation logic to validation constraints (see #14 there), to enable the many Field UX-tagged issues to move a lot faster thanks to not having to deal with brittle form validation anymore

IMHO the two next most important problems to tackle (which hopefully @phenaproxima will 🤞), with the most benefit for the ecosystem, would be to:

1. Create a type: langcode, which would benefit both type: config_object and type: config_entity, so would impact 100% of config.
2. Add validation constraints to type: _core_config_info, which again would benefit both type: config_object and type: config_entity, so would impact 100% of config.

Opened #3373653: Add a `langcode` data type to config schema to tackle:

Create a type: langcode, which would benefit both type: config_object and type: config_entity, so would impact 100% of config.

Opened #3374394: Add validation constraints to _core_config_info for:

Add validation constraints to type: _core_config_info, which again would benefit both type: config_object and type: config_entity, so would impact 100% of config

#3374394: Add validation constraints to _core_config_info and #3373653: Add a `langcode` data type to config schema have landed, and together they significantly improve on the status quo. Compared to #37:

Config types

  ℹ️   4.62% → 6.01% validatable (29 → 38 of 628 → 632 config types — excludes base types)
  ℹ️  25.94% → 28.90% average config type validatability
…
  ℹ️ 36.68% → 38.82% validatable property paths (1369 → 1470 of 3732 → 3787 property paths — this excludes property paths for base types)
…

Config entity types

  ℹ️   0.00% validatable (0 of 30 config types — excludes base types)
  ℹ️  24.05% → 31.54% average config type validatability
…
  ℹ️ 35.40% → 39.50% validatable property paths (194 → 220 of 548 → 557 property paths — this excludes property paths for base types)
…

Next up: continue the plan from #41 (#41.1 (working to get back to passing tests + RTBC after #3373653: Add a `langcode` data type to config schema landed!). In addition to expanding validatability the way I described in #41.3, @phenaproxima has been working on that same problem space (#42 + #43), and I just opened #3376794: `type: mapping` should use `ValidKeys: <infer>` constraint by default 🆕, which will bring a big boost.

P.S.: #41.2 has been RTBC for 2 weeks now! 👍

Update for the plan in #41 from ~1 month ago:

1. ✅ #3361534: KernelTestBase::$strictConfigSchema = TRUE and BrowserTestBase::$strictConfigSchema = TRUE do not actually strictly validate landed, plus all 4 of its follow-ups.
2. 👍 #3364506: Add optional validation constraint support to ConfigFormBase has been RTBC'd by the relevant subsystem maintainer.
3. ✅ #3341682: New config schema data type: `required_label` was re-scoped but is now RTBC, the original scope was handled by #3377030: Add validation constraint to `type: label`: disallow multiple lines
4. 👍 #3364109: Configuration schema & required values: add test coverage for `nullable: true` validation support is RTBC.
5. 🚀 #3364108: Configuration schema & required keys just became reviewable: #3364108-9: Configuration schema & required keys.
6. 🛑 #3324140: Convert field_storage_config and field_config's form validation logic to validation constraints is blocked on review from a subsystem maintainer
7. NEW: 🚀 #3379899: Follow-up for #3361534: config validation errors in contrib modules should cause deprecation notices, not test failures just became reviewable, and is a follow-up for #3361534.

and the two suggested next steps have also been completed:

1. #3373653: Add a `langcode` data type to config schema
2. #3374394: Add validation constraints to _core_config_info

Lots of other Configuration schema validation issues have landed besides those. For example: #3377055: Validate config schema compliance of Demo Umami and Minimal profiles, just like we do for Standard

A fun observation for #3361534: @jibran, maintainer of https://www.drupal.org/project/dynamic_entity_reference, reported over at #3361534-89: KernelTestBase::$strictConfigSchema = TRUE and BrowserTestBase::$strictConfigSchema = TRUE do not actually strictly validate that his module started failing tests against 11.x. That means he's probably the first (and likely the only! 😄😮) module maintainer who's running testa against Drupal 11. So, prioritized #3379899: Follow-up for #3361534: config validation errors in contrib modules should cause deprecation notices, not test failures, which now also is reviewable. Added to the numbered list above 👍

Many thanks to everyone involved:

• @borisson_
• @phenaproxima
• @smustgrave
• @penyaskito
• @znerol
• @dww
• @timplunkett
• @alexpott
• @catch
• @larowlan
• @longwave
• @bnjmnm
• @effulgentsia
• @lauriii

(I hope I didn't forget anyone! 🙈)

Issues created during drupalcon:

• Add validation constraints to olivero.settings
• Add validation constraints to filter.settings
• Add validation constraints to taxonomy.settings
• Add validation constraints to user.settings
• Add validation constraints to dblog.settings
• Add validation constraints to field.settings
• Add validation constraints to media.settings
• Add validation constraints to automated_cron.settings
• Add validation constraints to image.settings
• Add validation constraints to claro.settings

#3364108 is still rtbc and #3364109 needs a few more test fixes.

Just in case this is useful, there are three child issues marked as RTBC, that are waiting core maintainer action 😅.
Namely #3382581: Add new `EntityBundleExists` constraint at 2023-10-10, #3395616: Add validation constraints to image.settings at 2023-11-08, and #3395631: Add validation constraints to dblog.settings at 2023-10-30.

I kinda lost track of this meta 😅 But we've made a TON of progress. Most of the issues listed in #46 have landed. And thanks to #3422641: Add a "Validatable config" tests job to GitLab CI to help core evolve towards 100% validatability having landed yesterday, we won't be making backwards steps anymore, only forwards 🥳

As of today, https://project.pages.drupalcode.org/config_inspector/ looks like this:

Also, the original purpose of this very issue has been unblocked: see the green MR over at #2300677: JSON:API POST/PATCH support for fully validatable config entities! We absolutely can and should proceed there, even before every config entity type has been made validatable. We now have the necessary infrastructure! 🤘

Yesterday, I've created a new meta issue with a clearer, more holistic view of things: #3427641: [meta] Config validation for a more reliable Drupal + reliable Recipes from the start. It splits things 2-ways at a high-level: patterns to apply all across core (with meta/plan issues for those, since those things have dozens of child issues) vs missing infrastructure.

Let's continue there 🙏 This issue has served us well over the past decade, time for a fresh start for the new decade! 😄

Improving discoverability.