Expose validation constraints (and validatability %) in Config Inspector UI

FYI: https://www.drupal.org/project/ckeditor5 also uses validation constraints extensively!

And now Drupal core uses it (because CKEditor 5 was moved into Drupal core). The Group module is also adopting it over at #2925819: Ability to force using a scheme instead of scheme-relative URLs.

I've started building this! 🥳

I've got it working locally already. I'm building on top of #3359418: Expose validation constraint violations in Config Inspector UI and drush command. I'm starting out with Validatability % first, but that means I've got 95% of what we need in place.

Sneak preview:

While working on this, I've discovered two blocking core bugs, both of which I've added a work-around for:

1. #3360991: TypedData instances created by TypedConfigManager::createFromNameAndData() are incomplete
2. #3361034: Make 'sequence' Typed Data instances return type of 'sequence' instead of 'list'

1/4 done.

1. $ vendor/bin/drush config:inspect --only-error
    --------------------- ---------- ------------- ---------- 
     Key                   Status     Validatable   Data      
    --------------------- ---------- ------------- ---------- 
     editor.editor.cke4    1 errors                           
     media.settings        Correct    17%           1 errors  
     system.theme.global   Correct    38%           1 errors  
     tour.tour.cdn-ui      8 errors                           
     update.settings       Correct    8%            1 errors  
    --------------------- ---------- ------------- ---------- 
   

2. Nothing in core is 100% validatable, but when we start making more things validatable, this new flag will be really helpful:
   

   $ vendor/bin/drush config:inspect --only-error --invalidatability-is-error
    ----------------------------------------------------------- ---------- ------------- ---------- 
     Key                                                         Status     Validatable   Data      
    ----------------------------------------------------------- ---------- ------------- ---------- 
     automated_cron.settings                                     Correct    0%            Valid     
     automatic_updates.settings                                  Correct    20%           Valid     
     block.block.claro_breadcrumbs                               Correct    27%           Valid     
     block.block.claro_content                                   Correct    27%           Valid     
     block.block.claro_help                                      Correct    27%           Valid     
   …
     editor.editor.basic_html                                    Correct    69%           Valid     
   
   …
     system.menu.footer                                          Correct    40%           Valid     
     system.menu.main                                            Correct    40%           Valid     
     system.menu.tools                                           Correct    40%           Valid     
   …
     views.view.who_s_new                                        Correct    18%           Valid     
     views.view.who_s_online                                     Correct    17%           Valid     
    ----------------------------------------------------------- ---------- ------------- ---------- 
   

3. Let's look at only a few:
   

   $ vendor/bin/drush config:inspect --only-error --invalidatability-is-error --filter-keys=media.settings,system.theme.global
    --------------------- --------- ------------- ---------- 
     Key                   Status    Validatable   Data      
    --------------------- --------- ------------- ---------- 
     media.settings        Correct   17%           1 errors  
     system.theme.global   Correct   38%           1 errors  
    --------------------- --------- ------------- ---------- 
   

4. Let's add detail:
   

   $ vendor/bin/drush config:inspect --only-error --invalidatability-is-error --filter-keys=media.settings,system.theme.global --detail
    ------------------------------ --------- ------------- ----------------------------------------------------- 
     Key                            Status    Validatable   Data                                                 
    ------------------------------ --------- ------------- ----------------------------------------------------- 
     _core                          Correct   NOT                                                                
     _core.default_config_hash      Correct   NOT                                                                
     icon_base_uri                  Correct   NOT                                                                
     iframe_domain                  Correct   NOT                                                                
     oembed_providers_url           Correct   NOT                                                                
     media.settings:iframe_domain   Correct                 This value should be of the correct primitive type.  
     favicon                        Correct   NOT                                                                
     favicon.mimetype               Correct   NOT                                                                
     favicon.path                   Correct   NOT                                                                
     favicon.url                    Correct   NOT                                                                
     features                       Correct   NOT                                                                
     logo                           Correct   NOT                                                                
     logo.path                      Correct   NOT                                                                
     logo.url                       Correct   NOT                                                                
     system.theme.global:logo.url   Correct                 This value should be of the correct primitive type.  
    ------------------------------ --------- ------------- ----------------------------------------------------- 
   

This looks surprisingly simple but took me MULTIPLE DAYS to put together! 😱

#3359418: Expose validation constraint violations in Config Inspector UI and drush command landed! Now this patch will apply. Re-testing.

But "strict" already has a particular meaning in the config schema world: \Drupal\KernelTests\KernelTestBase::$strictConfigSchema:

  /**
   * Set to TRUE to strict check all configuration saved.
   *
   * @see \Drupal\Core\Config\Development\ConfigSchemaChecker
   *
   * @var bool
   */
  protected $strictConfigSchema = TRUE;

IMHO "validatability" is just fine. Yes, it's kinda like "cacheability" in how little it was used. But it's perfectly valid English. And there is prior use of it beyond Drupal — for example \Symfony\Component\HttpFoundation\Response::isValidateable().

How about --invalidatable-is-error or --not-validatable-is-error? 🤔😊

Now validatability is shown in the UI 👍

Next up: showing validation constraints.

Thanks to #14 I noticed that media.settings has the type uri yet is marked as not validatable. That is not correct.

The problem:

• interface UriInterface extends PrimitiveInterface → note that this does not inherit StringInterface
• yet class Uri extends StringData implements UriInterface
• which means that due to class StringData extends PrimitiveBase implements StringInterface…
• Uri implements StringInterface too 🤪

I didn't account for that in ::computeNodeValidatability(). Fixed.

Oops, #14 and #16 were missing the new class that #10 added.

Maybe "strict" is fine. 🤔 Because we'd eventually want \Drupal\KernelTests\KernelTestBase::$strictConfigSchema to actually validate things too.

So … perhaps this is even appropriate! 😄

I'll update the patch and open a core issue for doing exactly that.

+++ b/src/Commands/InspectorCommands.php
@@ -104,19 +128,36 @@ class InspectorCommands extends DrushCommands {
+          if ($detail) {
+            foreach ($raw_validatability->getValidatabilityPerPropertyPath() as $property_path => $is_validatable) {
+              // Skip validatable property paths.
+              if ($onlyError && $invalidatability_is_error && $is_validatable === TRUE) {
+                continue;
+              }
+              $rows[$property_path] = ['key' => $property_path, 'status' => $status, 'validatability' => $is_validatable ? dt('Validatable') : dt('NOT'), 'data' => NULL];
+            }
+          }
+          // Continue to validating the data: even with incomplete
+          // validatability that is valuable to check.

🐛 This means that first "detail" rows are generated for each property path … and then we continue on toward validating, which results in output like this:

$ vendor/bin/drush config:inspect --invalidatability-is-error --filter-keys=media.settings --detail
 ------------------------------------------ --------- ------------- ----------------------------------------------------- 
  Key                                        Status    Validatable   Data                                                 
 ------------------------------------------ --------- ------------- ----------------------------------------------------- 
  media.settings:_core                       Correct   NOT                                                                
  media.settings:_core.default_config_hash   Correct   NOT                                                                
  media.settings:icon_base_uri               Correct   NOT                                                                
  media.settings:iframe_domain               Correct   Validatable                                                        
  media.settings:oembed_providers_url        Correct   Validatable                                                        
  media.settings:standalone_url              Correct   NOT                                                                
  media.settings:_core                       Correct                 Valid                                                
  media.settings:_core.default_config_hash   Correct                 Valid                                                
  media.settings:icon_base_uri               Correct                 Valid                                                
  media.settings:iframe_domain               Correct                 This value should be of the correct primitive type.  
  media.settings:oembed_providers_url        Correct                 Valid                                                
  media.settings:standalone_url              Correct                 Valid                                                
 ------------------------------------------ --------- ------------- ----------------------------------------------------- 

👆 Each property path is listed TWICE: once for "validatability" detail, once for "data validity" detail.

That should of course be combined.

This is better:

$ vendor/bin/drush config:inspect --invalidatability-is-error --filter-keys=media.settings --detail
 ------------------------------------------- --------- ------------- ----------------------------------------------------- 
  Key                                         Status    Validatable   Data                                                 
 ------------------------------------------- --------- ------------- ----------------------------------------------------- 
  media.settings                              Correct   33%           1 errors                                             
   media.settings:_core                       Correct   NOT           Valid                                                
   media.settings:_core.default_config_hash   Correct   NOT           Valid                                                
   media.settings:icon_base_uri               Correct   NOT           Valid                                                
   media.settings:iframe_domain               Correct   Validatable   This value should be of the correct primitive type.  
   media.settings:oembed_providers_url        Correct   Validatable   Valid                                                
   media.settings:standalone_url              Correct   NOT           Valid                                                
 ------------------------------------------- --------- ------------- ----------------------------------------------------- 

I took the liberty of slightly changing how --detail works. Rather than showing only details, I'm now also keeping the row for the containing config. Because that gives you a valuable high-level perspective: the % of validatability and the number of errors. Given the increase in information provided, I think this is a reasonable change.

Also fixed the PHPLint error (which does not occur on PHP 8, hence I missed it).

I changed it to use --strict but ran into

$ vendor/bin/drush config:inspect --strict --filter-keys=media.settings --detail
PHP Fatal error:  Uncaught TypeError: Drush\Preflight\PreflightArgs::setStrict(): Argument #1 ($strict) must be of type bool, null given in /Users/wim.leers/core/vendor/drush/drush/src/Preflight/PreflightArgs.php:453
Stack trace:
#0 [internal function]: Drush\Preflight\PreflightArgs->setStrict(NULL)
#1 /Users/wim.leers/core/vendor/drush/drush/src/Preflight/ArgsPreprocessor.php(85): call_user_func(Array, NULL)

… turns out drush reserves this argument: \Drush\Preflight\PreflightArgs::isStrict() 🤷‍♀️

So --strict-validation it is!

Fixed a typo from #22 … and refactored the logic to support collecting all validation constraints.

This actually allowed me to delete code! 😅🥳 (Turns out I was computing certain things twice…)

No UI yet: not in Drush nor in the web UI.

Noticed a bug introduced in #16 while verifying that the output for #23 still matched that of earlier patch iterations (it did — until #16 as expected, which is how I spotted this bug) — I inverted the condition there but forgot about one of the documented edge cases: BooleanInterface. Correctly handling that again results in significantly higher percentage of validatability, because a lot of config schemas use type: boolean.

I do not yet see how the list of validation constraints can sanely be shown on the Overview or the List view. But I can see a way for the Tree view:

(Also fixed a missing use from #24 — splitting up in atomic commits allows for easy mistakes 🙈)

… but I really believe that such a bulleted list just does not give enough context. I think symbols can provide a lot of clarity here, without the need for adding a lot of additional words:

P.S.: I went all the way, and made this work well in RTL contexts too:

And now, the ability to list constraints using --list-constraints in Drush:

1. Let's look at my module's config:

   $ vendor/bin/drush config:inspect --filter-keys=cdn.settings
    -------------- --------- ------------- ------- 
     Key            Status    Validatable   Data   
    -------------- --------- ------------- ------- 
     cdn.settings   Correct   36%           Valid  
    -------------- --------- ------------- ------- 
   

2. Let's add detail:
   

   $ vendor/bin/drush config:inspect --filter-keys=cdn.settings --detail
    ----------------------------------------- --------- ------------- ------- 
     Key                                       Status    Validatable   Data   
    ----------------------------------------- --------- ------------- ------- 
     cdn.settings                              Correct   36%           Valid  
      cdn.settings:                            Correct   NOT           Valid  
      cdn.settings:_core                       Correct   NOT           Valid  
      cdn.settings:_core.default_config_hash   Correct   NOT           Valid  
      cdn.settings:farfuture                   Correct   NOT           Valid  
      cdn.settings:farfuture.status            Correct   Validatable   Valid  
      cdn.settings:langcode                    Correct   NOT           Valid  
      cdn.settings:mapping                     Correct   NOT           Valid  
      cdn.settings:mapping.conditions          Correct   NOT           Valid  
      cdn.settings:mapping.domain              Correct   Validatable   Valid  
      cdn.settings:mapping.type                Correct   NOT           Valid  
      cdn.settings:scheme                      Correct   Validatable   Valid  
      cdn.settings:status                      Correct   Validatable   Valid  
      cdn.settings:stream_wrappers             Correct   NOT           Valid  
      cdn.settings:stream_wrappers.0           Correct   Validatable   Valid  
    ----------------------------------------- --------- ------------- ------- 
   

3. Let's list the constraints too:
   

   $ vendor/bin/drush config:inspect --filter-keys=cdn.settings --detail --list-constraints
    ----------------------------------------- --------- ------------- ------- ----------------------------------------------------------------------------- 
     Key                                       Status    Validatable   Data    Validation constraints                                                       
    ----------------------------------------- --------- ------------- ------- ----------------------------------------------------------------------------- 
     cdn.settings                              Correct   36%           Valid                                                                                
      cdn.settings:                            Correct   NOT           Valid                                                                                
      cdn.settings:_core                       Correct   NOT           Valid                                                                                
      cdn.settings:_core.default_config_hash   Correct   NOT           Valid                                                                                
      cdn.settings:farfuture                   Correct   NOT           Valid                                                                                
      cdn.settings:farfuture.status            Correct   Validatable   Valid   Drupal\Core\Validation\Plugin\Validation\Constraint\PrimitiveTypeConstraint  
      cdn.settings:langcode                    Correct   NOT           Valid                                                                                
      cdn.settings:mapping                     Correct   NOT           Valid                                                                                
      cdn.settings:mapping.conditions          Correct   NOT           Valid                                                                                
      cdn.settings:mapping.domain              Correct   Validatable   Valid   Drupal\cdn\Plugin\Validation\Constraint\CdnDomainConstraint                  
                                                                               Drupal\Core\Validation\Plugin\Validation\Constraint\PrimitiveTypeConstraint  
      cdn.settings:mapping.type                Correct   NOT           Valid                                                                                
      cdn.settings:scheme                      Correct   Validatable   Valid   Drupal\cdn\Plugin\Validation\Constraint\CdnSchemeConstraint                  
                                                                               Drupal\Core\Validation\Plugin\Validation\Constraint\PrimitiveTypeConstraint  
      cdn.settings:status                      Correct   Validatable   Valid   Drupal\Core\Validation\Plugin\Validation\Constraint\PrimitiveTypeConstraint  
      cdn.settings:stream_wrappers             Correct   NOT           Valid                                                                                
      cdn.settings:stream_wrappers.0           Correct   Validatable   Valid   Drupal\cdn\Plugin\Validation\Constraint\CdnStreamWrapperConstraint           
                                                                               Drupal\Core\Validation\Plugin\Validation\Constraint\PrimitiveTypeConstraint  
    ----------------------------------------- --------- ------------- ------- ----------------------------------------------------------------------------- 
   

4. Great, now let's limit it to errors only:
   

   $ vendor/bin/drush config:inspect --filter-keys=cdn.settings --detail --list-constraints --only-error
    ----- -------- ------------- ------ ------------------------ 
     Key   Status   Validatable   Data   Validation constraints  
    ----- -------- ------------- ------ ------------------------ 
   

6. That listed nothing because the config schema is valid and all data is valid. So now let's enable the strict validation mode:
   

   $ vendor/bin/drush config:inspect --filter-keys=cdn.settings --detail --list-constraints --only-error --strict-validation
    ----------------------------------------- --------- ------------- ------- ------------------------ 
     Key                                       Status    Validatable   Data    Validation constraints  
    ----------------------------------------- --------- ------------- ------- ------------------------ 
     cdn.settings                              Correct   36%           Valid                           
      cdn.settings:                            Correct   NOT           Valid                           
      cdn.settings:_core                       Correct   NOT           Valid                           
      cdn.settings:_core.default_config_hash   Correct   NOT           Valid                           
      cdn.settings:farfuture                   Correct   NOT           Valid                           
      cdn.settings:langcode                    Correct   NOT           Valid                           
      cdn.settings:mapping                     Correct   NOT           Valid                           
      cdn.settings:mapping.conditions          Correct   NOT           Valid                           
      cdn.settings:mapping.type                Correct   NOT           Valid                           
      cdn.settings:stream_wrappers             Correct   NOT           Valid                           
    ----------------------------------------- --------- ------------- ------- ------------------------ 
   

I think/hope this proves that this all works great now. Let's switch to the example I used for showing the validatability before: media.settings:

$ vendor/bin/drush config:inspect --filter-keys=media.settings --detail --list-constraints --only-error --strict-validation
 ------------------------------------------- --------- ------------- ----------------------------------------------------- -------------------------------------------------------- 
  Key                                         Status    Validatable   Data                                                  Validation constraints                                  
 ------------------------------------------- --------- ------------- ----------------------------------------------------- -------------------------------------------------------- 
  media.settings                              Correct   43%           1 errors                                                                                                      
   media.settings:                            Correct   NOT           Valid                                                                                                         
   media.settings:_core                       Correct   NOT           Valid                                                                                                         
   media.settings:_core.default_config_hash   Correct   NOT           Valid                                                                                                         
   media.settings:icon_base_uri               Correct   NOT           Valid                                                                                                         
   media.settings:iframe_domain               Correct   Validatable   This value should be of the correct primitive type.   Drupal\Core\Validation\Plugin\Validation\Constraint\Pr  
                                                                                                                            imitiveTypeConstraint                                   
 ------------------------------------------- --------- ------------- ----------------------------------------------------- -------------------------------------------------------- 

That's the same command as the one above, just replacing cdn.settings with media.settings. Makes sense to me!

IMHO this is ready now. An idea for a follow-up could be to distinguish between inherited constraints vs local constraints — that would make it clear that for example the _core key (which is present in both cdn.settings and media.settings's Drush output) actually is of the type _core_config_info — so adding a validation constraint there would fix it everywhere.

Another follow-up would be to map the FQCNs to the equivalent types:

• \Drupal\Core\Validation\Plugin\Validation\Constraint\PrimitiveTypeConstraint would become PrimitiveType because of
  

   * @Constraint(
   *   id = "PrimitiveType",
   *   label = @Translation("Primitive type", context = "Validation")
   * )
  

• \Drupal\cdn\Plugin\Validation\Constraint\CdnDomainConstraint would become CdnDomain because of
  

   * @Constraint(
   *   id = "CdnDomain",
   *   label = @Translation("CDN domain", context = "Validation"),
   * )
  

• and so on

… but that's really just a nice-to-have compared to today, where this information is still painfully difficult to access.

RE: "strict" → created #3361534: KernelTestBase::$strictConfigSchema = TRUE and BrowserTestBase::$strictConfigSchema = TRUE do not actually strictly validate for core.

Good call!

I was not happy about that either.

Having worked on #3361534: KernelTestBase::$strictConfigSchema = TRUE and BrowserTestBase::$strictConfigSchema = TRUE do not actually strictly validate in the mean time, I think I know how to distinguish: Valid (when actually properly validatable) vs Correct primitive type (when no real validation constraints are present, only primitive type is being checked):

$ vendor/bin/drush config:inspect --filter-keys=system.theme.global --detail --strict-validation
 --------------------------------------------------------- --------- ------------- ----------------------------------------------------- 
  Key                                                       Status    Validatable   Data                                                 
 --------------------------------------------------------- --------- ------------- ----------------------------------------------------- 
  system.theme.global                                       Correct   53%           1 errors                                             
   system.theme.global:                                     Correct   NOT           Correct primitive type                               
   system.theme.global:_core                                Correct   Validatable   Valid                                                
   system.theme.global:_core.default_config_hash            Correct   Validatable   Valid                                                
   system.theme.global:favicon                              Correct   NOT           Correct primitive type                               
   system.theme.global:favicon.mimetype                     Correct   NOT           Correct primitive type                               
   system.theme.global:favicon.path                         Correct   NOT           Correct primitive type                               
   system.theme.global:favicon.url                          Correct   NOT           Correct primitive type                               
   system.theme.global:favicon.use_default                  Correct   Validatable   Valid                                                
   system.theme.global:features                             Correct   NOT           Correct primitive type                               
   system.theme.global:features.comment_user_picture        Correct   Validatable   Valid                                                
   system.theme.global:features.comment_user_verification   Correct   Validatable   Valid                                                
   system.theme.global:features.favicon                     Correct   Validatable   Valid                                                
   system.theme.global:features.node_user_picture           Correct   Validatable   Valid                                                
   system.theme.global:logo                                 Correct   NOT           Correct primitive type                               
   system.theme.global:logo.path                            Correct   NOT           Correct primitive type                               
   system.theme.global:logo.url                             Correct   Validatable   This value should be of the correct primitive type.  
   system.theme.global:logo.use_default                     Correct   Validatable   Valid                                                
 --------------------------------------------------------- --------- ------------- ----------------------------------------------------- 

and in the UI:

What do you think?

Adding a legend is trivial per https://symfony.com/doc/current/components/console/helpers/table.html#sp.... We could choose to add it at the top or bottom.

But … \Consolidation\OutputFormatters\FormatterManager::write()'s unavoidable call to \Consolidation\OutputFormatters\FormatterManager::validateAndRestructure() unfortunately forcefully requires every row to contain the columns that all other rows have.

After a LOT of searching and trial and error I found a way to make it work. But this only allows us putting the legend at the top. (Which is probably preferable anyway.) This is the result:

$ vendor/bin/drush config:inspect --filter-keys=media.settings --strict-validation --detail
 Legend for Data: 
  ✅❓  → Correct primitive type, detailed validation impossible.
  ✅✅  → Correct primitive type, passed all validation constraints.
 -------------------------------- --------- ------------- ------------------------------------- 
  Key                              Status    Validatable   Data                                 
 -------------------------------- --------- ------------- ------------------------------------- 
  media.settings                   Correct   43%           1 errors                             
   media.settings:                 Correct   NOT           ✅❓                                 
   media.settings:_core            Correct   NOT           ✅❓                                 
   media.settings:_core.default_   Correct   NOT           ✅❓                                 
  config_hash                                                                                   
   media.settings:icon_base_uri    Correct   NOT           ✅❓                                 
   media.settings:iframe_domain    Correct   Validatable   This value should be of the correct  
                                                           primitive type.                      
   media.settings:oembed_provide   Correct   Validatable   ✅✅                                 
  rs_url                                                                                        
   media.settings:standalone_url   Correct   Validatable   ✅✅                                 
 -------------------------------- --------- ------------- ------------------------------------- 

Note that the legend only appears when needed:

$ vendor/bin/drush config:inspect --filter-keys=media.settings --strict-validation
 ---------------- --------- ------------- ---------- 
  Key              Status    Validatable   Data      
 ---------------- --------- ------------- ---------- 
  media.settings   Correct   43%           1 errors  
 ---------------- --------- ------------- ---------- 

and

$ vendor/bin/drush config:inspect --filter-keys=media.settings --strict-validation --detail --fields=key,status,validatability
 ------------------------------------------- --------- ------------- 
  Key                                         Status    Validatable  
 ------------------------------------------- --------- ------------- 
  media.settings                              Correct   43%          
   media.settings:                            Correct   NOT          
   media.settings:_core                       Correct   NOT          
   media.settings:_core.default_config_hash   Correct   NOT          
   media.settings:icon_base_uri               Correct   NOT          
   media.settings:iframe_domain               Correct   Validatable  
   media.settings:oembed_providers_url        Correct   Validatable  
   media.settings:standalone_url              Correct   Validatable  
 ------------------------------------------- --------- ------------- 

I also made it stand out visually:

Finally, I made this work in the UI too:

AWESOME! 😊🥳🙏

I already started working on what I said in #27 and #28:

An idea for a follow-up could be to distinguish between inherited constraints vs local constraints — that would make it clear that for example the _core key (which is present in both cdn.settings and media.settings's Drush output) actually is of the type _core_config_info — so adding a validation constraint there would fix it everywhere.

+

Another follow-up would be to map the FQCNs to the equivalent types:

• \Drupal\Core\Validation\Plugin\Validation\Constraint\PrimitiveTypeConstraint would become PrimitiveType because of
  

   * @Constraint(
   *   id = "PrimitiveType",
   *   label = @Translation("Primitive type", context = "Validation")
   * )
  

• \Drupal\cdn\Plugin\Validation\Constraint\CdnDomainConstraint would become CdnDomain because of
  

   * @Constraint(
   *   id = "CdnDomain",
   *   label = @Translation("CDN domain", context = "Validation"),
   * )
  

• and so on

… but that's really just a nice-to-have compared to today, where this information is still painfully difficult to access.

Stay tuned!

Kept my promise, see #3364412: Improve display of validation constraints: constraint IDs instead of FQCNs and local vs inherited 😄