NotNullConstraintValidator conflates "required" with "non-empty" for lists and complex data; only config schema is exempted

Problem/Motivation

A required value answers three separate questions: is it present, may it be null, and may it be empty. JSON Schema keeps all three apart (required for presence, "null" in type for nullability, minItems/minProperties for emptiness). Drupal's required flag currently answers all three at once, and this issue is about the third: an empty collection is treated as if it were null. The first two (presence versus nullability, see #5) are a separate conversation - data definitions have no nullability concept at all.

Two pieces of the typed data system combine so that "required" means "non-empty" for every list and complex data value, with no way for a consumer to say otherwise:

1. TypedDataManager::getDefaultConstraints() adds NotNull whenever $definition->isRequired() - a presence property becomes a value constraint.
2. NotNullConstraintValidator::validate() coerces an empty ListInterface or ComplexDataInterface value to NULL before delegating to Symfony, so Drupal's NotNull behaves as "not empty" for collections. Symfony itself keeps these apart (NotNull is null-only; NotBlank catches empty), as does JSON Schema.

The coercion dates to #2343035: Upgrade validator integration for Symfony versions 2.5+, and it is the semantic Field API wants: a required field means "at least one item". But it leaks onto every other typed data consumer, and core has already acknowledged that once: #3364109: Configuration schema & required values: add test coverage for `nullable: true` validation support added a hardcoded exemption for config schema's ArrayElement, with the comment "TRICKY: ... configuration schema sees a substantial difference between an empty sequence/mapping and NULL." That fixed one subsystem by class check and left the conflation in place for everything else.

Where I'm hitting this issue. The Tool API contrib module validates tool inputs and outputs against a published JSON Schema, where a required map or list output legitimately holds an empty value (an empty result set). With output enforcement on, core's NotNull failed real tools with "This value should not be null." for values their schema allows (tool#3583040). Because the exemption is a hardcoded instanceof ArrayElement, a module using plain Map/ItemList cannot opt out; Tool API had to stop marking rebuilt list definitions required and re-implement the null-only half of the check itself - which in turn means its definitions under-report isRequired() to anything that introspects them.

Related: #3347033: NotNullConstraintValidator doesn't catch empty strings, only list/complex data types documents the mirror-image inconsistency (empty strings are NOT coerced). One interaction to account for: on maps, the coercion currently doubles as a stopgap for #3256536: Map data definition properties are not validated if root value is empty - an empty map's required properties are never visited by RecursiveContextualValidator, so the NotNull rejection of [] is the only thing catching a required property missing from an empty required map. Any consumer switching the option off for maps needs #3256536: Map data definition properties are not validated if root value is empty fixed or its own property walk; the Tool API workaround was scoped to lists for exactly this reason.

Steps to reproduce

$definition = \Drupal::typedDataManager()->createDataDefinition('map')->setRequired(TRUE);
$violations = \Drupal::typedDataManager()->create($definition, [])->validate();
// One violation: "This value should not be null." - for a present, empty map.

Proposed resolution

Make the empty-to-null coercion an option of Drupal's NotNull constraint rather than a hardcoded behavior plus a hardcoded exemption:

• Add an option (for example treatEmptyAsNull, default TRUE) to the NotNull constraint plugin. The validator coerces only when the option is on.
• getDefaultConstraints() keeps emitting plain NotNull (default on), so Field API behavior is unchanged.
• The ArrayElement carve-out becomes config schema passing treatEmptyAsNull: FALSE where it builds constraints, replacing the instanceof check.
• Consumers that mean JSON-Schema-style "required but may be empty" declare NotNull with the option off, and express minimums with Count - which is already how the schema side of such systems works.

Long-term direction. The option is step one of a correction, not a permanent flag. Once consumers (config schema, contrib) declare treatEmptyAsNull: FALSE where they mean it, a future major can flip the default and deprecate the coercion entirely, with Field API expressing its actual requirement explicitly (a dedicated constraint that ignores empty field items, since Count would count them; plain typed data can use Symfony's NotBlank or Count), at which point NotNull means exactly what it says and the option itself can be removed.

Alternatives considered: removing the coercion outright breaks required-field semantics everywhere; teaching getDefaultConstraints() about collections changes behavior for fields too. The constraint option is BC-clean by default.

Remaining tasks

• Agree on the option name and default.
• Patch validator and constraint plugin, replace the ArrayElement instanceof with the option, add kernel coverage for all four combinations (empty/null x option on/off).

User interface changes

None.

API changes

A new, optional constraint option; no behavior change without it.

Data model changes

None.

AI was used to assist in drafting this issue.