Problem/Motivation
Primitive data types describe their normalized output with the JsonSchema attribute, and Serializer::getJsonSchema() exposes it through the json_schema format. See the change record.
Nothing describes a map. ComplexDataNormalizer does not opt in to the json_schema format, so the serializer falls back to TypedDataNormalizer, which returns a placeholder with only a $comment. Validators ignore $comment, so any data passes. A map's property definitions already describe the object, and each property can already describe itself.
Steps to reproduce
$map = \Drupal::typedDataManager()->create(
MapDataDefinition::create()
->setPropertyDefinition('id', DataDefinition::create('integer')->setRequired(TRUE))
->setPropertyDefinition('label', DataDefinition::create('string'))
);
\Drupal::service('serializer')->getJsonSchema($map, []);
// ['$comment' => 'No schema is defined for property of type Drupal\Core\TypedData\Plugin\DataType\Map. ...']
Expected:
[
'type' => 'object',
'properties' => [
'id' => ['type' => 'integer'],
'label' => ['oneOf' => [['type' => 'string'], ['type' => 'null']]],
],
'required' => ['id'],
'additionalProperties' => FALSE,
]
Proposed resolution
Add MapSchemaNormalizer, which only supports the json_schema format and only the Map class itself. It builds an object schema from the same non-internal properties that ComplexDataNormalizer::normalize() iterates.
ComplexDataNormalizer stays out of the json_schema format. Field item normalizers extend it and add keys that aren't properties, such as target_uuid on entity reference items, so a schema built from properties would not match their output. Field items, and other Map subclasses, keep the fallback.
Decisions:
additionalPropertiesisFALSE.Map::setValue()accepts unknown keys, butnormalize()only outputs defined properties.- Field items are out of scope.
- Maps without property definitions, maps whose properties are all internal, and maps whose properties are named
0,1,2keep the fallback. None of them normalize to a JSON object with known keys. Numbered properties describe a list, which should useListDataDefinition. isRequired()maps to the parent'srequiredlist. Separating presence from nullability is #3625989.
Remaining tasks
- Address the MR review.
- Subsystem maintainer review.
- Decide whether to update the existing change record or write a new one.
User interface changes
None.
Introduced terminology
None.
API changes
New serializer.normalizer.map_schema service. Serializer::getJsonSchema() returns an object schema for maps with property definitions. Normalization output does not change.
Data model changes
None.
Release notes snippet
AI-Generated: Yes. I used Claude Code to draft this issue summary and to check it against the MR. I reviewed and edited the result.
Issue fork drupal-3627318
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #2
mglamanComment #3
mglamanComment #4
mglamanComment #5
mglamanComment #6
mglamanComment #9
dries commentedI took a crack at this and opened a merge request for this.
Disclaimer: I built this with the help of an LLM but reviewed everything extensively.
While working on this MR, I kept running into nuances: cases where a map didn't behave the way I expected. I wanted to share some of my learnings so any review goes faster, or you can correct me when I'm wrong.
The basics
A map is typed data for a set of named values. Think of it as a PHP array with string keys. Its definition can declare properties:
The map's value is the PHP array it contains, like
['title' => 'Hello', 'count' => 3], and its keys are that array's keys. When the definition declares properties, the keys are the property names.The serializer turns typed data into plain PHP arrays, which is called normalizing, and then into JSON as
{"title": "Hello", "count": 3}.The serializer can also describe data as a JSON Schema. However, before this MR, core could not describe maps as JSON Schema.
When core can't describe something as JSON Schema, it returns the fallback schema:
{"$comment": "No schema is defined for property of type ..."}. Clients and JSON validators ignore$commentand, maybe no small detail, any data passes validation.Every map gets the fallback today. The merge request adds
MapSchemaNormalizer, which gives the map above the following JSON Schema:So far so good, so now the nuances, in order of increasing head scratching:
Nuance 1: internal properties are left out
A definition can mark a property as internal, with
setInternal(TRUE). Internal properties are left out of the normalized data, and we leave them out of the JSON Schema as well. (@michaellander, I believe that is different from Tool API, but it felt more correct.)Nuance 2: a map without property definitions accepts any array
I didn't fully realize that you can have a map without property definitions. It makes sense now, but it's good to know that in this case, any array is accepted.
The challenge is that its keys aren't known in advance, and because of that, it gets/keeps the fallback schema. There is a test for that.
Nuance 3: properties named 0, 1, 2 become a JSON array
PHP has a function
array_is_list(), which calls an array a list "if its keys consist of consecutive numbers from 0 to count($array)-1" (see https://www.php.net/manual/en/function.array-is-list.php). Interestingly,json_encode()encodes such a list as a JSON array and any other array as a JSON object:[0 => TRUE, 1 => 'a']becomes[true,"a"]['0' => TRUE, '1' => 'a']also becomes[true,"a"], because PHP turns the keys'0'and'1'into the integers 0 and 1[0 => TRUE, 'key7' => 'a']becomes{"0":true,"key7":"a"}[]becomes[], not{}Long story short, some arrays are lists apparently, and lists gets encoded as JSON arrays. An object schema would reject that array, so I decided to keep the fallback schema. It's an edge case, but I added a test for it.
Strictly speaking, the fallback may not be the best we could do here. We know the map will be encoded as a JSON array, so we could describe it as one. Given that it is such an edge case, we could make this a follow-up issue, or try to include it later in this MR.
Nuance 4: field items are maps, but keep the fallback schema
Each value of an entity's field, like a node's author, is a field item, and field item classes extend
Map. But field items have normalizers of their own, which can add keys to the output that aren't properties. Yes, you might need to read that twice. A node's author, for example, outputs one property,target_id, but its normalizer also addstarget_type,target_uuidandurl.To be honest, I don't know why yet. And because of that, I decided that
MapSchemaNormalizerdescribes only theMapclass itself, and field items, like any other class that extendsMap, keep the fallback schema. The test's last case checks that. The key to understand is that this isn't a regression.For example, a node's author data is normalized to:
Before this change and after it, its JSON schema is:
So this changes nothing for field items: they get the fallback schema today and still do after this MR. I need to dig into this more, but we may be able to improve this in a follow-up issue. This MR provides us the foundation for that.
Hope that helps! With that, check out the code.
Comment #10
nicxvan commentedIt might be worth asking @bradjones1, he's the subsystem maintainer.
There are enough edge cases outlined here I'd be curious if the correct approach is cleaning them up or maintaining the current fallback behavior. L
Comment #11
mglamanThanks for this, Dries. I updated the issue summary to match the MR, and left a review on the MR.
On the nuances:
'0','1','2'is a list. Typed data hasListDataDefinitionfor that, and config schema hassequence. The fallback is the right answer for that definition, so I don't think it needs a new issue to describe it as a JSON array.additionalProperties: my summary had this wrong.Map::setValue()keeps unknown keys, butMap::getProperties()only builds properties from the definitions, sonormalize()never outputs them. The schema can set it toFALSE.Mapping
isRequired()torequiredlines up with #3625989, whererequiredmeans present and nullability gets its own flag. I added it as a related issue.@nicxvan agreed, @bradjones1 should weigh in. The edge cases are narrow, though: every one of them keeps today's fallback, so nothing gets worse.
Generated with the help of an LLM.
Comment #12
mglamanComment #13
dries commentedIn the same spirit as #3625989-22: NotNullConstraintValidator conflates "required" with "non-empty" for lists and complex data; only config schema is exempted, my last commit refactors the tests I added earlier.
As I wrote in #3625989:
The refactored tests stop comparing each schema with a hardcoded array written in the test. Instead, they use core's JSON Schema validator instead. This makes JSON Schema our contract and avoids drift in those hardcoded arrays.
To do: an empty person, with every value
NULLis left out until #3256536: Map data definition properties are not validated if root value is empty lands.