Problem/Motivation
While working on #3364108: Configuration schema & required keys and many other "config schema validation"-related issues, I've encountered a number of bugs in Drupal core's config schema.
The latest one is
filter_settings.filter_html:
type: filter
…
which makes no sense because:
# Filter with module and status.
filter:
type: mapping
label: 'Filter'
mapping:
id:
type: string
label: 'ID'
provider:
type: string
label: 'Provider'
status:
type: boolean
label: 'Status'
weight:
type: integer
label: 'Weight'
settings:
type: filter_settings.[%parent.id]
See how at the very end of that definition, there's type: filter_settings.[%parent.id]. So this is essentially a circular schema definition 😅
This is a bug that went unnoticed for almost a decade. It's becoming apparent now that we're starting to use more of the pre-existing config schema for richer validation.
👆 This was fixed in #3404431: Filter settings schema types are incorrect.
This issue uncovered one more occurrence of a circular config schema type reference, in type: layout_builder.section. See #21 for details.
Steps to reproduce
N/A
Proposed resolution
Add basic validation to config schema definitions: verify
- that the
typeis valid - that there's no circular reference
Remaining tasks
- ✅
Basic validation: verify at least thetypeis valid and that there's no circular reference. - ✅
Write tests
User interface changes
None.
API changes
API addition: Extension::getAbsolutePath() — change record
Data model changes
None.
Release notes snippet
None.
| Comment | File | Size | Author |
|---|---|---|---|
| #11 | 3401837-nr-bot.txt | 90 bytes | needs-review-queue-bot |
Issue fork drupal-3401837
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #3
wim leersComment #4
wim leersLOTS of failures like this:
👍
Comment #5
wim leersThe code needs refining, but I'd first like feedback from a subsystem maintainer before spending many more hours on this (this already took me 2–3 hours).
I would like an explicit +1 that this change would be welcomed.
Comment #6
wim leersComment #7
borisson_I'm not a subsystem maintainer, but I like this - it would make schema definitions more robust. I wonder if this is something that we need to add to TypedConfigManager, or somewhere else, but that is an implementation detail.
Comment #8
wim leersWhat would the other possible location be? 🤔
Maybe the
config.storage.schemaservice, which is used byTypedConfigManager? That would force it to happen at a lower level/earlier time, which may be better?Comment #9
phenaproximaComment #10
wim leersSee #5.
Comment #11
needs-review-queue-bot commentedThe Needs Review Queue Bot tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".
This does not mean that the patch needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.
Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.
Comment #12
wim leersComment #14
kostyashupenkoI don't think we need to add this tag. It just meant MR needs rebase and rebase is done
Comment #15
wim leers@kostyashupenko I know. But the point is that this MR should NOT chase HEAD. See #10 and #5.
Thanks for the rebase though! :D
Comment #16
borisson_#8, it has taken me a while to get back to this, I have thought this trough, I really wanted to suggest to put the validation of the schema in a new service/class, but I think it makes most sense to put it here, where we are adding a lot of the other validation passes as well.
Comment #17
wim leersFYI the actual fixes that this MR makes are also in #3404431: Filter settings schema types are incorrect. This issue aims to add infrastructure to prevent similar problems.
Comment #18
smustgrave commentedNot sure if still needs subsystem review.
But MR appears to be unmergable.
Comment #19
wim leers#3404431: Filter settings schema types are incorrect landed
Comment #20
wim leersAll feedback except one piece has been addressed, and tests are now passing!
Moving on to the last piece of feedback… which is also the only
@todoI had added 👍Comment #21
wim leersThe last piece of feedback/the
@todoI had added was to not just run this validation for thefilter_settings.filter_htmlconfig schema type, but for ALL config schema types.Doing this seemed to work fine locally, but caused 254 test failures on GitLab CI. They all look like this:
Turns out that #2942661: Sections should have third-party settings introduced the ability to define third party settings on Layout Builder Sections. So the
layout_builder.sectiontype got this addition:(Copy/pasted from
type: config_entity.) But this is wrong; it should have used the same pattern as for example field formatter settings (which are stored inEntityViewDisplayconfig entities akacore.entity_view_display.*.*.*) getting third party settings; they usedtype: field.formatter.third_party.[%key]. Similarly, here we needed:Why is this necessary? Because:
core.entity_view_display.*.*.*.third_party.layout_builder", which is only one of many possible third party settings on entity view display config entities — all config entities allow arbitrary third party settings using[%parent.%parent.%type].third_party.[%key]sectionskey that is a list of Layout Builder Sections (i.e.layout_builder.section)type: layout_builder.sectionusing that same[%parent.%parent.%type].third_party.[%key], the grandparent/two levels up (%parent.%parent) does not refer to the root of the config entity and uses its type (core.entity_view_display.*.*.*), but instead it refers to …layout_builder.sectionitself! 👈 this is the circular reference! ⚠️Next up: test coverage. This needs test coverage using a pattern similar to the one in
\Drupal\KernelTests\Config\Schema\MappingTest::testInvalidMappingKeyDefinition().Comment #22
wim leersTried to write tests.
The more representative test coverage pattern to copy is actually
\Drupal\Tests\ckeditor5\Kernel\CKEditor5PluginManagerTest::testInvalidPluginDefinitions().Writing test coverage for this is very complicated because of the hardcoded assumptions in
ExtensionDiscovery. See\Drupal\Tests\ckeditor5\Kernel\CKEditor5PluginManagerTest::mockModuleInVfs(), where I previously had to deal with this. 😬 To be continued tomorrow.Comment #23
phenaproximaThis is making more and more sense.
It's very obscure, complicated logic (not your fault...that seems to be the nature of the config schema system). I'm wondering if we can change the comments/method names in some places to make it as clear as possible to future developers (or just "future us") who need to come to grips with this stuff.
But the logic itself, I don't have any real complaints about.
Comment #24
wim leers🏓
It's not obscure, but it is complicated. To understand the logic, you have to understand how config schema type definitions. Once you understand that, it should no longer be obscure. And in fact, this logic should help make config schema in general less obscure for the next person 😅
Comment #25
phenaproximaHopefully I didn't hit a nerve :) I meant it's obscure because config schema itself is kind of obscure. Hopefully as it becomes more "useful" (and consistent!) thanks to this work, the obscurity will disappear.
Comment #26
wim leersHahaha, not at all — no worries! 😄 What you're saying in #25 is the same as what I said in #24. We agree that knowledge about the config schema system is obscure (hard-won) knowledge, that we're trying to make more accessible in this issue by removing some of it 👍
Comment #27
wim leersThe tests were hard to write — I was able to copy most of the pattern in
\Drupal\Tests\ckeditor5\Kernel\CKEditor5PluginManagerTest::mockModuleInVfs()but not everything. Which also led me to push #2961541 forward again — see #2961541-26: ExtensionDiscovery fails to scan the vfsStream filesystem for modules or profiles.Comment #28
wim leersAll feedback addressed 👍
Comment #29
wim leersComment #30
phenaproximaA couple of questions but I don't think I have any problems with this in general.
Although I think I understand the code here, I'm not really comfortable RTBCing it; it needs subsystem maintainer review anyway, so that's probably who should give it the green light.
Comment #31
wim leersRemaining feedback addressed!
@phenaproxima I'm hoping you can close 3 of the 4 threads 🙏 (one is a committer FYI and can remain open)
Comment #32
wim leersAddressed @alexpott's MR feedback. That resulted in an API addition (
Extension::getAbsolutePath()) and a change record: https://www.drupal.org/node/3420082.Comment #33
phenaproximaI'm very nearly out of complaints here. I think I only found one or two nits, plus a request for clarification that should be trivial.
I think someone else should give the official RTBC since I'm not an expert in the subject matter. But as far as I can tell, this is RTBC after my final comments are addressed.
Comment #34
wim leersComment #35
wim leers#3392903: Validate inputs of TypeResolver::resolveExpression(): only allow %parent, %type and %key landed!
I won't have time for this soon though … so unassigning.
Comment #37
bbralaRebased this issue, fixed some phpstan/phpcs issues. Hopefully fixed all remarks. Lets see how it runs on gitlab :)
Comment #38
eduardo morales albertiIn our case we use https://www.drupal.org/project/config_inspector to fix errors with schemas related to config
Comment #39
bbralaMade a follow up for the remaining comment. That same comment did say this is ok for now.
https://www.drupal.org/project/drupal/issues/3517417
Guessing subsystem maintiner would be Alex? Since that has been a while and it will probably go through alex when committing im going to set this to NR.
(test failure was unrelated)
Comment #40
borisson_There is a note in the MR about not having to deprecate a method because it was not shipped in a release, since then it has though, so do we need to do a proper deprecation now?
Comment #41
bbralaYeah you're right. We might as well, seems quite easy.
Comment #42
bbralaUpdated the code to deprecate the method instead of just removing it.
Comment #43
borisson_Looks good, thanks!
Comment #44
quietone commentedTrying for a better title.
Comment #45
alexpottI guess a year later I feel stronger that we should address this this way.
So looking at core and grepping for
root.*>getPath\(reveals a few places the new method can be used...So perhaps my instinct to add the method in 2024 was a good - let's do it. Especially since \Drupal\Core\Extension\Extension::__wakeup and \Drupal\Core\Extension\Extension::__sleep take care not to serialize the root.
Comment #46
alexpottWe need to add the new method to \Drupal\Core\Config\TypedConfigManagerInterface