[Prototype] Write a setting's rules once, in config schema, and use them for validation, forms and the published JSON Schema

Problem/Motivation

Most settings in Drupal are described in two or more places: in their config schema, their settings form, Tool API attributes, and more.

Take the maximum cache age setting on Drupal Core's Performance page (/admin/config/development/performance). Its config schema specifies the following in system.schema.yml:

max_age:
  type: integer
  label: 'Max age'

Its settings form PerformanceForm has this code:

$period = [0, 60, 180, 300, 600, 900, 1800, 2700, 3600, 10800, 21600, 32400, 43200, 86400];
$period = array_map([$this->dateFormatter, 'formatInterval'], array_combine($period, $period));
$period[0] = '<' . $this->t('no caching') . '>';
$form['caching']['page_cache_maximum_age'] = [
  '#type' => 'select',
  '#title' => $this->t('Browser and proxy cache maximum age'),
  '#config_target' => 'system.performance:cache.page.max_age',
  '#options' => $period,
  '#description' => $this->t('This is used as the value for max-age in Cache-Control headers.'),
];

Compare the two:

• The label differs: "Max age" vs "Browser and proxy cache maximum age".
• Only the form has a description.
• Only the form limits the value. The config schema accepts any integer, including a negative one. So config written without the form, by a recipe, a config import or code, can specify a negative value, and validation doesn't catch it.

It's not ideal, plus these gaps become increasingly painful the more Drupal interacts with external systems, which it does more and more. See two of my blog posts: One description, inside and outside Drupal and Drupal's role in agentic workflows for additional context.

Drupal Core's serialization module can generate JSON Schema for typed data values, including config values. Other software that sends data to Drupal through an API, like a JavaScript front end, an external workflow tool, or an AI agent, can check a value against a JSON Schema before it sends it. This is becoming the norm, if it isn't already.

For max_age, the generated JSON Schema is currently:

{
  "oneOf": [
    {
      "type": "integer"
    },
    {
      "type": "null"
    }
  ]
}

It basically specifies that max_age is an integer or null, but nothing more. (It allows null because system.performance doesn't have the FullyValidatable constraint.)

Needless to say, external software does not have access to Drupal's form specification. This means the specification should get to them through the generated JSON Schema. Long story short, we want Drupal to generate something like this for max_age:

{
  "title": "Browser and proxy cache maximum age",
  "description": "This is used as the value for max-age in Cache-Control headers, in seconds.",
  "type": "integer",
  "minimum": 0,
  "maximum": 31536000,
  "default": 0
}

Even where a config schema includes constraints today, the generated JSON Schema currently leaves them out. Take this example from Drupal Core's system.date:

first_day:
  type: integer
  label: 'First day of week'
  constraints:
    Range:
      min: 0
      max: 6

It currently generates {"type": "integer"} for its JSON Schema. Unlike max_age, it has no null option, which is good. However, we really want it to generate something like {"title": "First day of week", "type": "integer", "minimum": 0, "maximum": 6}, so software outside of Drupal knows we start numbering from 0, not 1 (as you do, of course).

Contributed modules like Tool API work around all these limitations with their own custom code. They turn constraints like Length, Range, Regex, Choice and Count into more detailed JSON Schema. As you can imagine, this is quite a bit of overhead for these modules, and it really belongs in Core in my opinion.

Continuing with our second example, system.date, its settings form, RegionalForm, basically repeats or duplicates the config schema:

$form['locale']['date_first_day'] = [
  '#type' => 'select',
  '#title' => $this->t('First day of week'),
  '#config_target' => 'system.date:first_day',
  '#options' => [
    0 => $this->t('Sunday'),
    1 => $this->t('Monday'),
    2 => $this->t('Tuesday'),
    3 => $this->t('Wednesday'),
    4 => $this->t('Thursday'),
    5 => $this->t('Friday'),
    6 => $this->t('Saturday'),
  ],
];

The keys of #options are the numbers 0 to 6 that the Range constraint allows. The weekday names also need to move to config schema, which has no way to give allowed values labels yet.

These gaps are widespread in Drupal Core and, by extension, in contrib. In the future, we'd like each setting's rules to live in its config schema: its label, its description, its validation constraints, and its allowed values with their labels. The form, the generated JSON Schema and every client would then read them from there. The MR in this issue shows how this could work end to end.

This work cuts across several parts of core: Typed Data and its validation constraints, the configuration system and its config schema, the serialization module, JSON Schema generation, and Form API. It will also touch Single Directory Components as they describe their props in JSON Schema. The prototype doesn't touch SDCs yet, but they won't be forgotten about. ;-)

Disclaimer: I'm using AI tools to help me learn and prototype. I will get it wrong in places, and so will AI. I'm not an expert on all these systems, and I'm learning a lot working on this. I've been really enjoying it so far. But getting it wrong is part of the point of this issue: it gives us something concrete to look at and react to.

As I said in my DrupalCon Rotterdam keynote: Drupal has more or less all the capabilities we need, but what is missing is "the last mile" that fully connects all the different parts. So I wanted to dig in more and see how all of this could fit together. The prototype in the MR is pretty straightforward right now. It does go end to end: from a constraint in config schema, to the JSON Schema Drupal generates, to the settings form.

It's straightforward because it builds on years of great work by many people in the community, including subsystem maintainers and core committers. I'm far from the first person to point this out.

Related work

For years, people have pointed out that a setting is described twice, once in its config schema and once in its form (#2913372: Allow forms to be defined in three segments: schema, UI, data, #2949888: Enhance config schema for richer default experiences). In fact, pieces and prototypes of a fix exist in various places:

• michaellander's #3622144: Data surfaces: a definition layer between Form API and storage proposes "data surfaces", a definition layer between Form API and storage. His core-gaps write-up (https://github.com/mlander/surface/blob/main/docs/core-gaps.md) lists what core is missing, step by step, and he ported Honeypot to show it working.
• fago's portable-schema (https://github.com/fago/portable-schema) describes data so that PHP and JavaScript validate it the same way. It also proposes an important rule that a generated JSON Schema must never reject a value Drupal accepts.
• Tool API uses attributes to describe a tool's input and output properties and generates the tool's JSON Schema from them. Example:
  

  #[ToolProperty(description: 'The year the album is from.', constraints: ['Range' => ['min' => 1900, 'max' => 2099]])]
  public function getYear(): ?int {
  

  A #[ToolInput] attribute can also name a config object instead, and the input's definition then comes from that config's schema.

• In #3477363: Add a way to automatically generate basic config forms, joachim and murz propose generating basic config forms from config schema.
• e0ipso's schema-forms-php (https://github.com/e0ipso/schema-forms-php) builds a Form API form straight from a JSON Schema: JSON Schema → form. This prototype goes from config schema → data definition → form and JSON Schema. To take a JSON Schema from outside Drupal, like a component's props, #3627345: No API to build data definitions from a JSON Schema would add JSON Schema → data definition, and the same form code would then build the form. Bi-directional support is beyond the scope of this prototype, but worth exploring more.
• mglaman's #3627330: Validation constraints do not contribute to the JSON Schema proposes that constraints add their limits to the JSON Schema Drupal generates.
• bradjones1's #3031367: Generate JSON schema for content entity types added JSON Schema generation for content entities to the serialization module.

Some of the ideas in the list above go in different directions. fago, michaellander, mglaman, e0ipso and many others have been doing so much great work on this. As part of Rosetta, I'm helping to bring it all together into one plan, and this prototype is a start.

Prototyped solution

This section provides a bit more detail on the prototyped solution to help anyone review the code and jump in.

The MR adds a new interface, JsonSchemaConstraintInterface, with one method, getJsonSchemaKeywords(). LengthConstraint, RangeConstraint and CountConstraint are its current implementations. Other constraints are left until we agree on the design/architecture.

When the serialization module generates a value's JSON Schema, we now add the "validation keywords" to the JSON Schema (e.g. "minimum": 0). We also added a value's label as title and its description as description. In other words, the MR improves the generated JSON Schema for first_day and max_age, consistent with the explanation above. I updated both in the MR as working examples to look at.

JSON:API builds its resources' JSON Schema with the serialization module, so the JSON:API output improved too.

I also modified ConfigFormBase so it uses the #title from the config value's label, #description from its description, and #maxlength, #min and #max from the constraints' keywords.

For backward compatibility, ConfigFormBase never overrides a property the form element sets itself. This allows us to gradually strip elements from form code. See the updated forms for first_day and max_age.

For max_age, the form code was:

$form['caching']['page_cache_maximum_age'] = [
  '#type' => 'select',
  '#title' => $this->t('Browser and proxy cache maximum age'),
  '#config_target' => 'system.performance:cache.page.max_age',
  '#options' => $period,
  '#description' => $this->t('This is used as the value for max-age in Cache-Control headers.'),
];

In the MR, max_age's config schema and form element read:

max_age:
  type: integer
  label: 'Browser and proxy cache maximum age'
  description: 'This is used as the value for max-age in Cache-Control headers.'

$form['caching']['page_cache_maximum_age'] = [
  '#type' => 'select',
  '#config_target' => 'system.performance:cache.page.max_age',
  '#options' => $period,
];

What's left in the form element is a select and its options. I believe this could/should move as well. It would get us basic form generation for free and simple settings form could go away entirely. See #3477363: Add a way to automatically generate basic config forms.

The first_day form code was simplified to:

$form['locale']['date_first_day'] = [
  '#type' => 'select',
  '#config_target' => 'system.date:first_day',
  '#options' => [
    0 => $this->t('Sunday'),
    1 => $this->t('Monday'),
    2 => $this->t('Tuesday'),
    3 => $this->t('Wednesday'),
    4 => $this->t('Thursday'),
    5 => $this->t('Friday'),
    6 => $this->t('Saturday'),
  ],
];

Once config schema can give allowed values a label, first_day's weekday names could come from the schema too, and this form could pretty much go away after #3477363.

A helper in JsonSchemaKeywordsTest, assertPublishedSchemaMatchesValidation(), checks fago's rule that a generated JSON Schema must never reject a value Drupal accepts. This felt useful for development, but we may not need it long-term. I left it in the MR for now.

How you can help

1. Software outside Drupal can see a setting's rules

A front end, an integration or an AI agent can check a value before sending it, instead of learning from a failed save.

• Add support for Choice. It probably has to wait for #3634887: JSON Schema gets a second null option when a #[JsonSchema] attribute allows null without a type.
• Add support for NotBlank. It should publish minLength: 1 or minItems: 1.
• Add support for Regex using pattern. Challenge: PHP patterns and JavaScript patterns used by clients don't always match. Example: PHP-only syntax like \z breaks a JavaScript validator, and in PHP, $ also matches before a final newline.
• File an issue for the empty email address: system.site:mail accepts an empty email address, but its "format": "email" rejects it.
• Review the null fixes: #3627331: Nullability is inconsistent across JSON Schema normalizers, #3634887: JSON Schema gets a second null option when a #[JsonSchema] attribute allows null without a type and #3634179: JSON:API generates invalid JSON Schemas for field types with a single optional property.
• Write the JSON Schema for mappings (#3627318: ComplexDataNormalizer does not provide a JSON Schema for complex data) and lists (#3627328: ListNormalizer does not provide a JSON Schema for lists), and fix asking for a schema by class name (#3627338: Serializer::getJsonSchema() throws a TypeError when passed a class name).
• Make the serializer read the constraints inside a ComplexData constraint. Until then, string fields publish no maxLength, because StringItem keeps its Length there.
• Try adding constraints to more config schema. For example, #3427641: [meta] Config validation for a more reliable Drupal + reliable Recipes from the start and #3437608: Add validation constraints to system.performance.
• Once the keywords land, switch Tool API and other modules that map constraints themselves to core's JSON Schema.

2. A developer describes a setting once

Today a setting's label, description and limits are written in its config schema and again in its form. There are many things we can update, but I'd like to suggest the following first:

• Try moving the site slogan's '#maxlength' => 255 into config schema, as a Length on system.site:slogan.
• Try converting GD's jpeg_quality form to #config_target, so its Range is written only in config schema.
• Propose in #2949888: Enhance config schema for richer default experiences how config schema gives allowed values a label, so first_day's weekday names can move out of the form into configuration.
• Settle in #3625989: NotNullConstraintValidator conflates "required" with "non-empty" for lists and complex data; only config schema is exempted whether "required" means not null or not empty, so ConfigFormBase knows when to set #required.
• Figure out if potx should collect config schema descriptions for translation. Until then, a description moved into config schema, like max_age's, shows only in English on a translated site. Gábor might like this one.

3. Software outside Drupal can change settings

A client reads a setting's schema, changes the setting through an API, and gets Drupal's own error message back when a value fails.

• Propose where core serves a config value's JSON Schema, in the serialization module or JSON:API, and at which URL.
• Review #3426508: Generate JSON Schema for config entity types, which generates JSON Schema for config entity types.
• Review #2300677: JSON:API POST/PATCH support for fully validatable config entities, which adds JSON:API POST and PATCH for config entities.
• Propose how a client shows Drupal's error messages: fago's catalog of messages with IDs, or michaellander's one English string per message.
• Build the reader that turns a JSON Schema back into data definitions in #3627345: No API to build data definitions from a JSON Schema.