Field data for multiple cardinality fields are not populated on entity load

Bumping priority.

Bumping to critical, I made the change described in the issue summary into an MR, as mentioned it exactly matches what's done with single cardinality fields.

Multiple field loading is a bit different between main/11.x and 11.4.x now, but I think this logic is more or less exactly the same so it may even apply to all three branches.

Since it’s critical should tests be deferred?

Going to go on a limb and say we should probably land this one and maybe follow up test coverage? Only because it's critical and seems like a bad one too. Would never suggest other wise.

The test as described in the IS does not seem that difficult to write, but at the very least if we're going to commit this without it, someone should confirm with manual testing.

The test as described in the IS does not seem that difficult to write

Famous last words.

Gave a shot at writing the test, but I could not reproduce a failing case, either on main or 11.4.x. Even if I create the original node with "default_langcode" explicitly set to FALSE, the field values load. I've pushed up what I have so far to a new MR: https://git.drupalcode.org/project/drupal/-/merge_requests/16618.

I've had difficult reproducing the issue manually as well. Here are the steps taken (on the main branch, IIRC):

1. Install Drupal standard
2. Install language and content_translation modules
3. Add Spanish as a second language, leave English as default
4. Create a content type
5. Make content type translatable
6. Add 1 single cardinality translatable text field, 1 single cardinality untranslatable text field, 1 multiple cardinality translatable text field, and 1 multiple cardinality untranslatable text field
7. Make language selector available on node form
8. Create new node in Spanish and add values for all fields and save
9. Add an English translation and save

Multiple cardinality values always appear as expected, even if I edit the untranslatable field from the form for the translation.

@tomaston or anyone who has encountered this: Can you provide detailed steps to reproduce this issue? Ideally using core only starting with a Standard profile installation, but if contrib is involved, please list what contrib modules need to be installed.

I can reproduce this on a real site running Drupal 11.4.4, and the proposed change fixes the problem.

My case is slightly different from the current issue summary: the affected field is a **translatable, unlimited-cardinality taxonomy term entity reference field**, rather than a non-translatable field.

### Environment

* Drupal core: 11.4.4
* Database: MariaDB
* Entity type: node
* Bundle: `model`
* Field: `field_model`
* Field type: `entity_reference`
* Target type: `taxonomy_term`
* Cardinality: unlimited (`-1`)
* Translatable: `true`
* Node language: `en`

This database was originally a Drupal 10 database. The site was upgraded/migrated to Drupal 11. I later restored a newer copy of the Drupal 10 database underneath the Drupal 11 codebase and ran the Drupal 11 configuration/database update process.

The same node and field work correctly when the database is used with the Drupal 10 site.

### Example affected entity

Node:

```
nid: 63665
vid: 104857
bundle: model
langcode: en
```

The field's database row is intact:

```
SELECT *
FROM node__field_model
WHERE entity_id = 63665;
```

returns:

```
bundle: model
deleted: 0
entity_id: 63665
revision_id: 104857
langcode: en
delta: 0
field_model_target_id: 141
```

The corresponding revision table also contains the value:

```
SELECT *
FROM node_revision__field_model
WHERE entity_id = 63665
AND revision_id = 104857;
```

returns the same `field_model_target_id = 141`.

The taxonomy term with TID 141 also exists.

The node's current/default translation metadata is:

```
nid: 63665
vid: 104857
langcode: en
default_langcode: 1
revision_translation_affected: 1
```

### Drupal 10 behavior

On the Drupal 10 site:

```
drush php:eval '
$n=\Drupal\node\Entity\Node::load(63665);
var_dump($n->get("field_model")->getValue());
'
```

returns:

```
array(1) {
[0]=>
array(1) {
["target_id"]=>
string(3) "141"
}
}
```

### Drupal 11.4.4 behavior

Against the same content in Drupal 11.4.4:

```
drush php:eval '
$n=\Drupal\node\Entity\Node::load(63665);
var_dump($n->get("field_model")->getValue());
'
```

returns:

```
array(0) {
}
```

This is not an entity-cache issue; `resetCache()`, `loadUnchanged()`, and loading revision 104857 directly all return an empty `field_model`.

The field storage configuration is valid and matches the Drupal 10 site:

```
type: entity_reference
settings:
target_type: taxonomy_term
cardinality: -1
translatable: true
custom_storage: false
```

The field storage UUID and bundle field UUID are also identical between the Drupal 10 and Drupal 11 sites.

### Drupal 11.4.4 does retrieve the row from SQL

I enabled Drupal's database query logger while loading the node.

Drupal 11.4.4 generates this query for the multi-cardinality field:

```
SELECT
node_field_data.*,
node_field_data.langcode AS node_field_data__langcode,
node__field_model.field_model_target_id AS field_model_target_id,
node__field_model.delta AS field_model_delta
FROM node_field_data node_field_data
INNER JOIN node__field_model node__field_model
ON node__field_model.entity_id = node_field_data.nid
AND node__field_model.langcode = node_field_data.langcode
AND node__field_model.deleted = 0
WHERE node_field_data.nid IN (...)
```

Executing the equivalent query manually returns:

```
nid langcode field_model_target_id delta
63665 en 141 0
```

So the database query successfully retrieves the value. The value is lost afterward while Drupal is populating the entity.

The final field item list is:

```
Field class: Drupal\Core\Field\EntityReferenceFieldItemList
Count: 0
Is empty: YES
Raw value: array(0)
First item: NULL
```

Other non-translatable entity-reference fields on this same node load normally.

### Proposed patch fixes the issue

I applied the proposed `default_langcode` guard to the Drupal 11.4.4 version of `loadMultipleCardinalityFields()`.

For 11.4.4 the relevant change is:

```
- if ($this->langcodeKey && isset($default_langcodes[$value_key]) && $row[$base_langcode_alias] != $default_langcodes[$value_key]) {
+ if ($this->langcodeKey && empty($row[$this->defaultLangcodeKey]) && isset($default_langcodes[$value_key]) && $row[$base_langcode_alias] != $default_langcodes[$value_key]) {
```

After applying that change, rebuilding caches, and running exactly the same test:

```
drush php:eval '
$n = \Drupal\node\Entity\Node::load(63665);
print_r($n->get("field_model")->getValue());
'
```

now returns:

```
Array
(
[0] => Array
(
[target_id] => 141
)
)
```

I then reverted the manual core edit, packaged the same one-line change as a Composer patch, had Composer reinstall Drupal core 11.4.4 and apply the patch, rebuilt caches, and repeated the test. It still correctly returns `target_id => 141`.

So in this database the proposed fix reproducibly changes the field from **0 loaded items to the correct stored taxonomy reference**.

One potentially important difference from the current issue description is that `field_model` is configured as `translatable: true`. I also temporarily changed the field to `translatable: false`, rebuilt caches, and the field still loaded empty before applying the core patch.

I would be happy to provide additional database state or run targeted queries against this affected database if that would help identify the exact data condition needed for an automated regression test.

A test should cover loading a non-translatable, multi-value field on an entity whose default-translation record's langcode differs from the computed default langcode, asserting the values are returned.

That paragraph in the IS 'Proposed Solution' means we need an automated test. Adding the tag.

I don't see how the proposed fix does anything.

We are adding an additional guard on empty($row[$this->defaultLangcodeKey]). But $row is created by the database query that is earlier in the method, and $this->defaultLangcodeKey (aka "default_langcode") is never added as a column in the query as far as I can see. So the empty() clause will always be true?

Can you reproduce on a fresh install of core? Do you have any core patches installed?

@bgelhard If possible, can you answer the following:

• Is the site multilingual? If so, does the node in question have translations?
• Are there other fields on the content type? If so, what cardinality and are they translatable?
• Can you show DB query results for the entire rows associated with the node (nid 141 looks like?) in the node, node_field_data, node__field_model tables, and also for the other corresponding field tables for the content type?

Thanks!

@godotislate:

The site is not multilingual; English (en) is the only configured language. Node 63665 has no translations. We do use String Overrides for runtime string replacement, but not content translation.

For node 63665:
vid: 104857
langcode: en
default_langcode: 1
revision_translation_affected: 1

The content type has many fields, including several unlimited-cardinality fields, but field_model is the only custom field marked translatable:

field_model
type: entity_reference
target: taxonomy_term
cardinality: -1
translatable: YES

There are several other unlimited-cardinality entity-reference fields, but they are all non-translatable and load correctly with and without the patch. For example, `field_implementation_requirement` is non-translatable, unlimited-cardinality, and Drupal correctly loads all 9 values for this same node.

**Affected database row**

node__field_model:

bundle: model
deleted: 0
entity_id: 63665
revision_id: 104857
langcode: en
delta: 0
field_model_target_id: 141

Before the patch, Drupal 11.4.4's generated SQL successfully retrieved field_model_target_id = 141, but...

Node::load(63665)->get('field_model')->getValue()

... returned an empty array.

After applying the proposed patch from this issue the exact same call returns...

target_id => 141

So a potentially important aspect of this reproduction is the site has only one language, but the unlimited-cardinality field where we saw the issue is configured as translatable.

I'm happy to provide the complete node/node_field_data rows or rows from the other field tables if those would help with the automated test.

The suggested fix in the description worked for me also after applying and clearing cache.

Diff file wasn't working for me Drupal 11.4.5, attached working patch if it's useful for anyone.

@joe huggans if you're running into this too, are you able to answer any of the questions in #18?

BACKGROUND

The problem appeared after upgrading to Drupal 11.

I investigated a specific affected content type on our site called resource.

CURRENT LANGUAGE CONFIGURATION

The site is not currently multilingual.

Language module: Disabled
Content Translation module: Disabled
Configured languages: English only
Default language: en

HISTORICAL LANGUAGE CONFIGURATION

However, the site was previously multilingual.

Repository history shows that until 2 December 2024:

- The Language and Content Translation modules were enabled.
- English, German, Spanish, and Dutch were configured.
- Content Translation was enabled for the resource node bundle.

Drupal translation was subsequently removed.

This historical multilingual configuration appears relevant, although the affected nodes do not currently have translations.

REPRESENTATIVE AFFECTED NODE

I used node 27 from the resource bundle as a representative example.

The node has no translations.

Entity language: en
Translation languages: en

CURRENT NODE BASE ROW

Table: node

Node ID: 27
Revision ID: 6956
Type: resource
Language code: en

CURRENT NODE REVISION ROW

Table: node_revision

Node ID: 27
Revision ID: 6956
Language code: und
Revision user ID: 79
Revision timestamp: 1724144384
Revision log: NULL
Default revision: 1

CURRENT NODE FIELD DATA ROW

Table: node_field_data

Node ID: 27
Revision ID: 6956
Type: resource
Language code: en
Published status: 1
Title: Hidden
User ID: 1
Created timestamp: 1540821278
Changed timestamp: 1724144384
Promoted: 1
Sticky: 0
Default language: 1
Revision translation affected: 1

CURRENT NODE FIELD REVISION ROW

Table: node_field_revision

Node ID: 27
Revision ID: 6956
Language code: en
Published status: 1
Title: Hidden
User ID: 1
Created timestamp: 1540821278
Changed timestamp: 1724144384
Promoted: 1
Sticky: 0
Default language: 1
Revision translation affected: 1

SIGNIFICANT LANGUAGE STATE

The significant database values are:

node.langcode = en

node_revision.langcode = und

node_field_data.langcode = en

node_field_data.default_langcode = 1

node_field_revision.langcode = en

node_field_revision.default_langcode = 1

There is only one actual translation, and its data row is explicitly marked as the default translation.

However, the corresponding revision base row contains the language code und.

FIELDS ON THE CONTENT TYPE

All configurable fields on the resource bundle are currently non-translatable at the bundle field-definition level.

The affected unlimited-cardinality fields are:

AFFECTED UNLIMITED-CARDINALITY FIELDS

Field: field_attachment
Cardinality: Unlimited
Field translatable: No
Storage translatable: Yes

Field: field_author
Cardinality: Unlimited
Field translatable: No
Storage translatable: Yes

Field: field_contains_attachment_types
Cardinality: Unlimited
Field translatable: No
Storage translatable: Yes

Field: field_languages
Cardinality: Unlimited
Field translatable: No
Storage translatable: Yes

Field: field_organiser_
Cardinality: Unlimited
Field translatable: No
Storage translatable: Yes

Field: field_resource_type
Cardinality: Unlimited
Field translatable: No
Storage translatable: Yes

Field: field_tags
Cardinality: Unlimited
Field translatable: No
Storage translatable: Yes

FIELD AND STORAGE TRANSLATABILITY

The distinction between field-definition and field-storage translatability appears important.

loadMultipleCardinalityFields() tests the bundle field definition using isTranslatable(). This returns FALSE for the affected fields.

SINGLE-CARDINALITY FIELDS

The content type also has these single-cardinality, non-translatable fields:

- body
- field_archive
- field_black_box_text
- field_has_attachments
- field_link
- field_meta_tags
- field_override_icon
- field_publication_date
- field_recommended
- field_reuse
- field_use_video_attachment_thumb

These single-cardinality fields load correctly.

loadSingleCardinalityFields() already checks default_langcode before assigning a non-default language key.

ROWS IN THE AFFECTED FIELD TABLES

All the rows below have the following properties:

Bundle: resource
Deleted: 0
Entity ID: 27
Revision ID: 6956
Language code: en

Table: node__field_attachment
Delta: 0
Value column: field_attachment_target_id
Value: 79

Table: node__field_attachment
Delta: 1
Value column: field_attachment_target_id
Value: 80

Table: node__field_author
Delta: 0
Value column: field_author_target_id
Value: 225

Table: node__field_contains_attachment_types
Delta: 0
Value column: field_contains_attachment_types_target_id
Value: youtube_video_a_

Table: node__field_contains_attachment_types
Delta: 1
Value column: field_contains_attachment_types_target_id
Value: attachment_presentation

Table: node__field_languages
Delta: 0
Value column: field_languages_target_id
Value: 44

Table: node__field_organiser_
Delta: 0
Value column: field_organiser__target_id
Value: 19

Table: node__field_resource_type
Delta: 0
Value column: field_resource_type_target_id
Value: 14

Table: node__field_tags
Delta: 0
Value column: field_tags_target_id
Value: 15

Table: node__field_tags
Delta: 1
Value column: field_tags_target_id
Value: 21

The corresponding revision field tables contain identical rows for revision 6956.

DATABASE VALUES COMPARED WITH LOADED VALUES

Despite the stored rows, without the proposed patch both the current entity and the revision load every affected field as empty.

field_attachment
Database items: 2
Loaded items: 0

field_author
Database items: 1
Loaded items: 0

field_contains_attachment_types
Database items: 2
Loaded items: 0

field_languages
Database items: 1
Loaded items: 0

field_organiser_
Database items: 1
Loaded items: 0

field_resource_type
Database items: 1
Loaded items: 0

field_tags
Database items: 2
Loaded items: 0

With the proposed default_langcode guard, both load() and loadRevision() return the expected item counts:

field_attachment: 2
field_author: 1
field_contains_attachment_types: 2
field_languages: 1
field_organiser_: 1
field_resource_type: 1
field_tags: 2

WHY THE ROWS APPEAR TO BE DROPPED

The initial entity query takes this value from node_revision:

langcode = und

loadFromDedicatedTables() therefore records und in $default_langcodes.

The multiple-cardinality query subsequently reads the authoritative data row with these values:

langcode = en
default_langcode = 1

The unpatched code compares en with und. Because they differ, it assigns the field row to the language key en instead of x-default.

The code then discards the row because the bundle field definition is non-translatable.

The proposed default_langcode guard prevents this misclassification when the row itself states default_langcode = 1. This matches the existing behavior in loadSingleCardinalityFields().

SUGGESTED REGRESSION TEST

A regression test could cover all of the following:

1. A revision base row using und.

2. A corresponding data row using en with default_langcode = 1.

3. A non-translatable, unlimited-cardinality field.

4. Both normal entity loading and loadRevision().

Thank you, @joe huggans for that very detailed report. Using that, I was able to produce and automated test case that fails in the test-only job https://git.drupalcode.org/project/drupal/-/jobs/11911914 and is fixed on 11.4.x by the proposed change: https://git.drupalcode.org/project/drupal/-/merge_requests/16938

The test has language installed as well as the en language config entity. There's a node type with single and multiple cardinality fields that are both translatable and untranslatable (from the test results, not sure that matters), and a node created of that type.

When language is uninstalled, the en language config entity is deleted, which causes NodeStorage::clearRevisionsLanguage() to change the langcode to und for all entries in the node_revision table that match the deleted langcode (en). From there, after loading the entity again, the field data for the multiple cardinality nodes are not populated.

However, since the loading of multiple cardinality fields has been refactored for 11.5/12.0, the change in the original MR against main does not pass the test: https://git.drupalcode.org/project/drupal/-/merge_requests/16468. This is likely a result of the changes from #3608184: Load multiple cardinality fields with a smaller result set and will need more investigation.

Title and IS could use some work based on latest findings.

Re #25 and #26 could the existing fix be applied to 11.4.x branch but postpone fix for 11.5/12.0.

Updated title and IS.

So, one thing I think we can do is prevent NodeStorage::clearRevisionsLanguage() from setting the langcode to und in node_revision for all rows that are in the default langcode. The best place for this logic is probably in NodeEntityHooks::configurableLanguageDelete(), so that it looks like this:

   #[Hook('configurable_language_delete')]
   public function configurableLanguageDelete(ConfigurableLanguageInterface $language): void {
-    // On nodes with this language, unset the language.
-    \Drupal::entityTypeManager()->getStorage('node')->clearRevisionsLanguage($language);
+    if ($language->getId() !== \Drupal::languageManager()->getDefaultLanguage()->getId()) {
+      // On nodes with this language, unset the language.
+      \Drupal::entityTypeManager()->getStorage('node')->clearRevisionsLanguage($language);
+    }
   }

This won't change anything though for deletions of other configured languages, nor will it address anything for existing node_revision rows that were already changed to und.

I'm RTBCing #25 for Drupal 11.4 - since this is a critical issue I think we should break our 'only backport' policy then leave this open for 11.5/main but obviously try to fix it before either of those are released.

Re #31 godoislate, which of the branches mentioned in #33 do you want to apply the change to?

Pushed a fix to MR!16468 for main/11.x. The fix was assisted by GPT 5.6.

loadMultipleCardinalityFields() does too much and takes way too many arguments for my liking, maybe we can refactor this elsewhere.

Re #35 in #33 catch was RTBTC'ing #25 only for 11.4. I think we could put back to RTBTC for #25 but keep working on 11.5/main starting with #35.

I think the priority is to fix critical bug in 11.4 asap.

MR!16468 needs performance test updates, but otherwise looks good.

I think #31 and refactoring loadMultipleCardinalityFields() can be follow ups, if we want to do them.

edited: completely mistaken comment. Rebasing and updating performance tests now.

main MR looks good now too. I think between @catch, @longwave, and me reviewing and working on separate parts of both MRs, I can RTBC this.

Committed and pushed e7ee50849bb to 11.4.x. Thanks!

This means we can release 11.4.6 with this fix included.

Leaving open for the fixes on main and 11.x.

#31 and the refactor both sound like good follow-ups. The original method had some of the highest cyclomatic complexity of any method in core until we split it up, but the split up version is not really much better either.

I only handled performance test updates and turning a patch into an MR, didn't actually write any code here, so I think I'm fine to commit the main/11.x work.

Opened #3620626: Data integrity issues with deleting languages and node revisions for the data integrity issue.

Also opening an 11.x backport MR since there are performance test conflicts there.

Committed/pushed the 11.x backport too.

Adding the needs followup tag since still need an issue for refactoring ::loadMultipleCardinalityFields() but I think the single cardinality method is as bad or worse, and we might want to tackle them together, and that indecision was enough to stop me immediately opening an issue.

Stubbed #3620705: Refactor to reduce complexity in loadSingleCardinalityFields() and loadMultipleCardinalityFields() for #48, so that we have the follow up in place. I've created the issue to handle both single and multiple cardinality fields, but it can be split later if we feel that works better.

@catch and @godotislate Still unfixed.

Which version of core are you testing with? What are the steps to reproduce the issue?