Problem/Motivation

After cache-rebuild via drush (only!), field labels are displayed in wrong language. Debugging shows this is due to cache pollution, see below.

(Please refrain from adding "me too" comments if you have wrong field label language under different curcumstances, to keep this issue focused.

(This has serious repercussions on automated deploy with no known workaround, so setting major.)

Steps to reproduce

(Not yet tested if that is enough, maybe more conditions are involved.)
- Languages: English (default), German (DE)
- Interface text language detection: Only select URL or URL+Session (not checked: "Customize Content language detection to differ from Interface text language detection settings" setting)
- Have some content types and fields with translated label
- (Clear cache in the UI => All fine.)
- Clear cache via drush cr => Some field labels are erroneously displayed in english (should be german).

Detection method "Selected language" which is set last and always on, is set to German.

Once we enable "Customize Content language detection" and only select "Interface" the error does not occur although it results in the same language selection as above. Also if we select "Browser" as Interface detection method, the error does not occur.

Debugging shows
- drush sqlq "select * from cache_discovery where cid='entity_bundle_field_definitions:node:event:de'" => Directly after drush cr and with no browser request, \Drupal\Core\Entity\EntityFieldManager::getFieldDefinitions stores field definitions for interface language=DE (german), that carry untranslated labels.
- Some more digging shows, that \Drupal\Core\TypedData\TypedDataManager::getPropertyInstance has a static cache (without cacheability or language awareness) that may be involved here.
- Backtrace shows that in cache rebuild, that definitions are used by e.g. views (for views data) and search_api (for datasource data).

We conclude that in the process of building the field definitions, the language system is somehow in an inconsistent state, where fixed-language-negitiation happened (as field definitions are built for InterfaceLanguage=DE), but config overrides for DE are NOT applied. (We don't grok the bootstrap system well enough to debug that further.)

What happens is that - while field definitions are cached by current language - config language overrides for that language are not taken into account for either base field overrides or configurable fields. In Drush the "current" language is always the default language, because currently no language negotation takes place for Drush, see #3544395: Current user missing from language negotiator outside of request context. Thus, no language overrides are ever applied.

(We may pass the buck between drupal language system and drush bootstrap, but let's coordinate here.)

Proposed resolution

Set the config override language to the current language when loading base field overrides and bundle fields (which ends up loading configurable fields).

Remaining tasks

???

User interface changes

None

API changes

None

Data model changes

None

Release notes snippet

TBD

Issue fork drupal-3221375

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

geek-merlin created an issue. See original summary.

geek-merlin’s picture

Issue summary: View changes
geek-merlin’s picture

Title: Wrong language field labes, only after drush cr » Wrong language field labes, (only) after drush cr
geek-merlin’s picture

This may or may not be involved.

geek-merlin’s picture

Issue summary: View changes
anruether’s picture

Title: Wrong language field labes, (only) after drush cr » Wrong language field labels, (only) after drush cr
anruether’s picture

Issue summary: View changes
anruether’s picture

Issue summary: View changes
anruether’s picture

Issue summary: View changes
anruether’s picture

Issue summary: View changes
anruether’s picture

Issue summary: View changes
heine’s picture

Title: Wrong language field labels, (only) after drush cr » Wrong language field labels after drupal_rebuild()

I see this as well after a rebuild via the rebuild script (/core/rebuild.php) meaning drush is not to blame.

The site has two languages, "en" and "nl" with "en" the default language. I can reproduce the issue with "Selected language" with the value "nl" as the only negotiator for the interface language. There are no settings for the content language.

Our workaround in ci: drush php-eval "\Drupal::service('plugin.cache_clearer')->clearCachedDefinitions();".

ayalon’s picture

We had the same issue and I found cause. But unfortunatly I have no explanation for the phenomen.

It happens if you have enabled the "LanguageNegotiationBrowser".

If you default language is NOT english and you run a drush cr inside the LanguageNegotiationBrowser the language is detected from the Symfony request and this is "en-us" resulting in a detected language "en".

This language detection somehow corrupts the cache. I did not figure out, how this happens.

If you skip the LanguageNegotiationBrowser when running drush cr from CLI, thi bug disappears.

Attached you find a patch.

geek-merlin’s picture

Thanks @ayalon for that excellent debugging work. I did not test that patch, but the resoning and the patch looks thorough.

> If you default language is NOT english and you run a drush cr inside the LanguageNegotiationBrowser the language is detected from the Symfony request and this is "en-us" resulting in a detected language "en".

Do you grok where that (fake) "symfony request" is created? I'd say we should fix that request to not prefer a language.

ayalon’s picture

My main question is, if you have enabled Language Negotiation by Browser in your environment. If not, my solution will not fix your issue.

I debugged the drush command "drush cr" with xDebug. You can set a breakpoint inside LanguageNegotiationBrowser.php and inspect the request.

I did not figure out, where this "Symfony" Request is mocked but I think it is done outside of the Drupal code. So it is almost impossible to fix because it would affect a lot of dependancies.

ayalon’s picture

StatusFileSize
new1.02 KB

Updated patch with correct path

geek-merlin’s picture

#16:
> My main question is, if you have enabled Language Negotiation by Browser in your environment. If not, my solution will not fix your issue.

Ugh, thanks for reminding me that i (should) have contextual information on this. I cross-checked out internal notes and we do NOT use browser negotiation. (Also i remember that i was quite sure this is a symptom of #2276749: TypedDataManager prototyping might cache incorrectly).

So yes, what you describe is a different issue indeed (with similar symptoms).

(Side note: 'HTTP_ACCEPT_LANGUAGE' => 'en-us' is added in \Symfony\Component\HttpFoundation\Request::create ad default, so i'd propose to open a drush issue to override that in \Drush\Boot\DrupalBoot8::bootstrapDrupalSiteValidate)

kndr’s picture

Issue tags: +Needs manual testing, +Bug Smash Initiative, +Needs tests
StatusFileSize
new51.19 KB

I have the same situation with "drush cr" in one of my installation. It wasn't easy to reproduce that but I managed to. I would update the issue summary with my steps but I don't know if this is the same case.

Steps to reproduce:

  1. Install a fresh Drupal site with a standard profile
  2. Login as admin
  3. Go to /admin/modules and install "Language and "Interface Translation" modules
  4. Go to /admin/config/regional/language and add the "German" language
  5. Go to /admin/config/regional/language/detection/selected and select "German", Save configuration
  6. Go to /admin/config/regional/language/detection/url and set 'en' prefix for English and no prefix for German.
  7. Go to /en/admin/config/regional/language/detection and press "Save settings"
  8. Go to /en/admin/structure/menu/manage/main and press "Add link"
  9. Enter the "Search" text in a "Menu link title" field
  10. Enter the "/search" text in a "Link" field
  11. Press "Save" button
  12. Run the terminal and execute "drush cr"
  13. Go to /user/1/edit
  14. You can see that the label and description of the field "Picture" are not translated.

Label for Picture field is not translated

Note:
Setting a Xdebug breakpoint (with a condition: $cid == 'entity_bundle_field_definitions:user:user:de') in a \Drupal\Core\Cache\UseCacheBackendTrait::set() function is very helpful while debugging "drush cr". The stack trace shows that menu links play some role in a cache pollution. The interesting point here is that if you don't create a 'search' menu link then the label of the Picture field is translated correctly to German. What is more interesting not every path in a menu link causes the problem.

kndr’s picture

The last patch didn't pass the automated test because it needs a patch as well. Could you test the MR !1361 patch manually? I'm curious if it is working in your cases. Your feedback is valuable before pushing this issue forward.

nanak’s picture

I've stumbled upon the same issue, and after a few hours of debuging, I don't think the language negotiation system itself is to blame (unless it actually is a different issue).

According to me, part of the problem lies in changes in the drupal_flush_all_caches() method introduced when #2160091: drupal_rebuild() rebuilds container twice, since drupal_flush_all_caches() also rebuilds it has been committed (this commit: https://git.drupalcode.org/project/drupal/-/commit/6e84b5797c2693b63fc32...), which now conditionnaly rebuilds the container, based on the type of the $kernel variable: KernelInterface or NULL.

The other part of the problem comes from #2650434: Clearing cache via UI in translated language resets config translation of field labels to default language. This introduces a new event, DrupalKernel::CONTAINER_INITIALIZE_SUBREQUEST_FINISHED, which is dispatched only when the container gets rebuilt when there's an actual Request object into the RequestStack: https://git.drupalcode.org/project/drupal/-/blob/9.3.x/core/lib/Drupal/C....

Now, the only core subscriber for this event is the LanguageRequestSubscriber https://git.drupalcode.org/project/drupal/-/blob/9.3.x/core/modules/lang..., whose purpose is to set the language used by the LanguageConfigFactoryOverride service, which is later used to load the right translation for entity field labels.

In short, here's what happens in drush cr before #2160091: drupal_rebuild() rebuilds container twice, since drupal_flush_all_caches() also rebuilds it got commited:
- drush cr calls drupal_rebuild()
- In drupal_rebuild, a Kernel is created, the container is initialized
- During the container initialization, a first check to know if the CONTAINER_INITIALIZE_SUBREQUEST_FINISHED should be dispatched. As the request has not been attached yet to the stack, the condition is not met, and the event is not dispatched.
- The request is pushed to the request stack in DrupalKernel::preHandle()
- In drupal_rebuild(), a few lines later, drupal_flush_all_caches() is called, which will clear all caches, invalidate the container anew, and rebuilds it in \Drupal::service('kernel')->updateModules($modules, $files);
- The container initialization runs again, but this time, the request is found in the stack, so the CONTAINER_INITIALIZE_SUBREQUEST_FINISHED event is dispatched, the language gets negotiated and set in the LanguageConfigFactoryOverride service for later use.
- When the entity field definitions are first accessed and cached, they are loaded and cached in the right language.

Now, here's what happens after it got commited:
- Steps 1 to 4 are identical
- drupal_flush_all_caches($kernel) is still called, but carry a new parameter: the $kernel that got created a few lines before.
- In drupal_flush_all_caches, several things have changed, but the most noticeable in our case is that the unconditional \Drupal::service('kernel')->updateModules($modules, $files) call has been replaced with a $kernel->rebuildContainer(), which is only called if $kernel is not in instance of DrupalKernel.
- In this case, the container has already been instanciated, so there isn't a second check to dispatch the CONTAINER_INITIALIZE_SUBREQUEST_FINISHED. Hence, the default language remains english
- When the entity field definitions are later accessed and cached (EntityFieldManager::getFieldDefinitions), they are loaded in the default LanguageConfigFactoryOverride language (en), but cached in the current negotiated language cid (de, fr, whatever).

As for why this do not occurs during a UI cache clear, it's because PerformanceForm::submitHandler calls drupal_flush_all_caches() with no $kernel parameter, forcing the container to be rebuilt (with an attached request obviously, triggering the event)

The solution for this may be as simple as a PHP_SAPI === 'cli' check in drupal_flush_all_caches() to trigger or not the container rebuild, but I doubt that is a proper solution, as it would basically undo the work that's been done in #2160091: drupal_rebuild() rebuilds container twice, since drupal_flush_all_caches() also rebuilds it

leksat’s picture

@kndr your patch works for me. I have no idea whether it's a correct fix (Drupal core is so complex nowadays) but the patch does its job. Field labels and descriptions are back 🎉

My setup:
- English (default) and German languages.
- The only language detection method enabled: "Selected language", set to German

kndr’s picture

@Lekstat I'm glad to hear that. You are right, Drupal core is complex. It's really hard to figure out how to solve this issue properly. @nanak made a really impressive research to show the big picture of that problem.

nanak’s picture

@nkdr the patch do work, but is rather a workaround than a proper solution. I wouldn't be surprised if other config entities were affected.
Locally, I've patched drupal_flush_all_caches() by adding a PHP_SAPI === 'cli' condition for the container to be rebuilt, but IMHO, the actual problem needs to be solved elsewhere (as the container still has to be rebuilt twice in order to have the CONTAINER_INITIALIZE_SUBREQUEST_FINISHED event dispatched)

kndr’s picture

@nanak Yes, I'm afraid so. Grasping the whole problem and all dependencies is out of my skills :) I count on smart, experienced developers who will be able to fix that.

leksat’s picture

StatusFileSize
new532 bytes

Confirming. @kndr's patch helped with the fields, but then I found other broken config entities.

Attaching the @nanak's workaround.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

andypost’s picture

sralton’s picture

@Leksat Thanks for adding the workaround as patch.
The patch helps me too with the fields, but I'd be interested in the broken config entities you found.

tim-diels’s picture

Patch in #27 works for me. The MR did not work.
I had the issue when clearing the cache through Drush and the facets were in default language instead of the render language.
The facets are Taxonomy labels.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

geek-merlin’s picture

Title: Wrong language field labels after drupal_rebuild() » Wrong language field labels after `drush cr` because of Drush language negotiation
Related issues: +#3336537: ConfigOverrideLanguage gets set to CurrentLanguage too late, leading to hard to debug wrong entity labels

FTR: I am quite sure that there are multiple reasons that this can happen.

IIRC this issue hass settled on drush cr and language negotiation.

I found another one of them in the related issue.

hchonov’s picture

Status: Active » Needs review
StatusFileSize
new604 bytes

We just experienced this and these are our steps to reproduce it:
1. Have a multilingual site with EN and DE.
2. Set EN as the default language.
3. Configure interface and content language detection to be based on URL.
4. Configure that for DE the path prefix is / and for EN it is /en.
5. Place breakpoints in
5.1. \Drupal\Core\Entity\EntityFieldManager::getFieldStorageDefinitions()
5.2. \Drupal\language\Config\LanguageConfigFactoryOverride::loadOverrides()
6. Execute drush cr with xdebug enabled
7. Observe how the entity field manager now uses "de" for building the cache ID since without passing url to drush the url negotiator returns "de".
8. Observe how when it goes to load the overrides the language config factory override uses "en" to load the overrides since its constructor takes the site's default language.

Clearing the caches over the UI no matter if on the EN or DE path prefixes does not seem to be causing issues. It breaks only when drush is involved into the cache rebuild.

I also noticed that LanguageRequestSubscriber does not get triggered at all as no KernelEvents::REQUEST event gets dispatched during drush cr.

In contradiction to #32 for me the MR solution worked but the PHP_SAPI check did not work.

Inspired by both approaches I am providing a mixed alternative that hopefully should be then covering all the cases for everyone and not only the ones we are observing in the entity field manager.

Status: Needs review » Needs work

The last submitted patch, 35: 3221375-35.patch, failed testing. View results

andypost’s picture

Thanks for #35 details, it looks exactly the same reason as discussed in #2806009: Installing a module causes translations to be overwritten

sleitner’s picture

Status: Needs work » Needs review

Patch failed only temporarily

smustgrave’s picture

Status: Needs review » Needs work

This was tagged for tests which appear to still be needed. Question with #2806009: Installing a module causes translations to be overwritten being committed did it fix the issue here at all?

Don't think it's related but there is also #3329066: Creating a new translation may delete translations with drafts may be worth mentioning.

jose reyero’s picture

Reading through this issue, just some thoughts for now...

1. It looks like we have two different issues here:

* Drush not triggering / respecting language negotiation / not settling on a language early enough
* Bad caching, not respecting language either or mixing config language and caching language.

2. Which language should Drush select? Not clear for me... is there a request / a URL / a language prefix ... ? Since this may quickly get really confusing I think we should aim at settling on a sensible default, also something that won't mess with deployments on different environments... which is I think the main use of drush... This should be IMO the Site Default language, and that's it. Easy and clear.

3. Whatever the final patch is, triggering language negotiation mid-request doesn't look like a good idea to me... So either we set it during initialization or we don't.... Again, 2) may be a sensible default.

4. So... I see two ways to fix this issue: A) Proper language / initialization with Drush to (whatever makes sense) or B) Fix caching, aka.. make config language consistent with caching language (whatever it is) or maybe skip persistent cache.

jose reyero’s picture

I've done some more tests with this one and Drupal 10.1.x...

As @nanak #22 mentions, behavior is changed after cache rebuilding was fixed. But we can reproduce it with the setup in the issue description and forcing the field rebuilding with drush, try:

drush cr && drush php-eval "\Drupal::service('entity_field.manager')->getFieldDefinitions('node', 'article');"

Some more debugging shows also the already mentioned mismatch #35 in getFieldDefinitions(), when running with drush, while current language is the one set as custom language (de), the config override language is the default one (en). So we can see why the patch here, which fixes that mismatch works https://git.drupalcode.org/project/drupal/-/merge_requests/1361

However, I don't think that is the best place to switch configOverrideLanguage... or maybe just if we switch it on a method call, we better switch it back before returning, but not sure about this...

What I think we need to do is fixing the internal consistency of getFieldDefinitions() using current language bug calling other methods that get configuration that may not be using 'current language', but 'config override language' instead...

And as a general rule, if we want some consistent API, there shouldn't be any function that does language dependent caching without ensuring anyting it calls is using the same language...

jose reyero’s picture

Thinking a bit more...

* There's no easy way to use language consistently through the API, different languages (current, configuration) are retrieved from the container everywhere... that's dependency injection I guess ...
* Inconsistencies happen at other levels too. String translation is not properly initialized with drush either.

// This one uses default language
drush php-eval "print t('User name');"

// This one triggers language negotiation and uses defined language.
drush php-eval "print t('User name', [], ['langcode'=> \Drupal::languageManager()->getCurrentLanguage()->getId()]);"

So yes, just aiming for consistency we need to fix the language initialization, not only config override language, bug locale language too.

And this is where everything happens, but not when running drush:

// class Drupal\language\EventSubscriber\LanguageRequestSubscriber
  private function setLanguageOverrides() {
    $this->negotiator->setCurrentUser($this->currentUser);
    if ($this->languageManager instanceof ConfigurableLanguageManagerInterface) {
      $this->languageManager->setNegotiator($this->negotiator);
      $this->languageManager->setConfigOverrideLanguage($this->languageManager->getCurrentLanguage());
    }
    // After the language manager has initialized, set the default langcode for
    // the string translations.
    $langcode = $this->languageManager->getCurrentLanguage()->getId();
    $this->translation->setDefaultLangcode($langcode);
  }
jose reyero’s picture

StatusFileSize
new836 bytes

This patch is a simple and rough version of #40 / 4B: Just use default language for drush - and CLI - *always*

It will fix the issue for most people, while also keeping consistent string translations. Just keep in mind the language when running drush will be *consistent* but maybe not the one you like.... as it's always the site default language.

(Not intended for committing, of course, just setting to needs review to trigger testing and see what happens)

szeidler’s picture

Experiencing the issue as well. Basically breaks every deployment, because of `drush cr`. In our case also the "Follow the user's language preference." language detection mode is affected.

Patch #35 has been solving it.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

heikkiy’s picture

We are experiencing this also during deployments. At first it seemed like the patch from issue #35 works and after applying the patch, drush cr seemed to set the language correctly. But it still seems there is a difference when running cache clear with drush and through the UI.

In our case it's also notable that without the cache clear in the UI not only field labels are in wrong language but it can also display some other content like taxonomy term labels in wrong language.

For example today we noticed that some taxonomy term values were in English language when viewing the site in Finnish. After clearing the caches in the UI it changed the language correctly to Finnish.

At the moment we have a deployment process which uses drush deploy during the deployment.

It seems like this has something to do with the language negotation because we started experiencing the issues after we changed some of the language negotiation settings and changed the Selected language to fallback to Finnish. Our site's default language is English. We have enabled the URL language detection from the language prefix and also the Selected language negotiation is set to fallback to Finnish. Other language negotiations are disabled.

Here is our current language.negotiation.yml with the site experiencing the issue.

_core:
  default_config_hash: uEePITI9tV6WqzmsTb7MfPCi5yPWXSxAN1xeLcYFQbM
session:
  parameter: language
url:
  source: path_prefix
  prefixes:
    en: en
    fi: fi
    sme: sme
    smn: smn
    sms: sms
    '': null
  domains:
    en: ''
    fi: ''
    sme: ''
    smn: ''
    sms: ''
selected_langcode: fi

Here is the language.types.yml example:

_core:
  default_config_hash: dqouFqVseNJNvEjsoYKxbinFOITuCxYhi4y2OTNQP_8
all:
  - language_interface
  - language_content
  - language_url
configurable:
  - language_interface
negotiation:
  language_content:
    enabled:
      language-interface: 0
    method_weights:
      language-content-entity: -9
      language-url: -8
      language-session: -6
      language-user: -4
      language-browser: -2
      language-interface: 9
      language-selected: 12
  language_url:
    enabled:
      language-url: 0
      language-url-fallback: 1
  language_interface:
    enabled:
      language-url: -20
      language-selected: -15
    method_weights:
      language-user-admin: -16
      language-url: -20
      language-session: -19
      language-user: -18
      language-browser: -17
      language-selected: -15

We also have another site with a similar problem but there we are trying to also follow the user's profile to set the site's language so that might play some part in the problem because language-user-admin is enabled.

Here is the language.negotiation.yml from that site:

_core:
  default_config_hash: J6QBFdEYFecHpT05_8KmHuTn9k9ELLdiLJO-fQS1nDk
session:
  parameter: language
url:
  source: path_prefix
  prefixes:
    en: en
    fi: fi
    '': null
  domains:
    en: ''
    fi: ''
selected_langcode: fi

And also the language.types.yml:

_core:
  default_config_hash: dqouFqVseNJNvEjsoYKxbinFOITuCxYhi4y2OTNQP_8
all:
  - language_interface
  - language_content
  - language_url
configurable:
  - language_interface
  - language_content
negotiation:
  language_content:
    enabled:
      language-url: -19
      language-selected: -14
    method_weights:
      language-content-entity: -20
      language-url: -19
      language-user: -18
      language-session: -17
      language-browser: -16
      language-interface: -15
      language-selected: -14
  language_url:
    enabled:
      language-url: 0
      language-url-fallback: 1
  language_interface:
    enabled:
      language-user-admin: -19
      language-url: -18
      language-selected: -15
    method_weights:
      language-user: -20
      language-user-admin: -19
      language-url: -18
      language-session: -17
      language-browser: -16
      language-selected: -15

Reading through comments for example from #43 I do understand the problem if recognizing the correct language when caches are cleared with drush and setting it to default language makes sense. But this does make it hard for example for automated deployments which are usually running drush cr multiple times for example with drush deploy. And after deployments and viewing for example content under /fi/ language prefix results in some field and term labels being in wrong language. As said, clearing the caches from the UI probably fixes the issue because then language negotiation works and it can probably get the correct language from the URL prefix but this does mean an extra step in deployments and content is showing wrong language after each deployment.

liam morland’s picture

Part of patch in #9 this related issue is similar to the patch in #35 of the present issue.

heikkiy’s picture

One detail to add is that we are currently using both the patch #35 from this issue and the patch #9 from issue #3114824.

At least with the patch from this issue seemed to give me some results. In my local environment when I was clearing caches with drush without the patch from this issue, it seemed like the language for labels was in wrong language. With the patch from this issue it seemed to work correctly when I cleared caches with drush and then immediately viewed the page in the correct language.

But in our production environment there are still cases where we get client error reports that the labels are in wrong language. Not 100% understanding all the mechanism around this issue but my guess would be that it might be also related when the actual node gets cached for the first time. So in my test cases it was first viewed with the correct language prefix so it cached correctly. But if someone for example with a wrong administration interface language views the same node for the first time, could this result in some cache pollution situation that it doesn't anymore use the correct language for from the prefix?

Currently it seems like our language negotiation configs should prefer language-url based on the above configurations but I wouldn't be surprised if there was some weirdness with language negotiation, fallback language and default language given how complex the language negotiation logic can be.

biancaradu27’s picture

StatusFileSize
new792 bytes
heikkiy’s picture

We discussed this issue in Slack and I tested the latest patch from #3406929: Configuration being imported by the ConfigImporter sometimes has stale original data against 10.2.6 to solve the issue. At least in our staging environment which was suffering from this issue, the patch seems to now solve it momentarily after deployment. But after a while the language was again mixed where the default English language was showing in the Finnish site. Manual drush cr fixes the issue as usual. Our symptom was that taxonomy term and select list values in frontend were showing in wrong language after Drush deployment. I was also able to remove the patch from #3114824: Add multilingual support for caching basefield definitions without it having a visible regression.

joseph.olstad’s picture

Issue tags: -Needs manual testing
heikkiy’s picture

Unfortunately my comment from #51 was not the final verdict. We have still seen this issue after deployments. Mostly the problem still happens with the multisite where term names and field labels are in wrong language after deployment. Everytime a manual cache clear from the UI fixes the issue. It seemed like the patch from #3406929: Configuration being imported by the ConfigImporter sometimes has stale original data might have even made the problem more visible because with that patch we were also seeing wrong labels in the default site when usually they are limited to the multisite instance.

I'll report more findings if we are able to track down the issue further. Of course any debug or test instructions that we can run and report here would help.

yevko’s picture

I'll report more findings if we are able to track down the issue further. Of course any debug or test instructions that we can run and report here would help.

Heikki, I can see that both sites of yours have their default language as something else than English. Have you already tried changing it to English? My colleagues said that this change fixed the translation issue. Meaning:

1. default_langcode: en in system.site.yml
2. selected_langcode: site_default in language.negotiation.yml
3. Make sure that every configuration is in English

I need to make this work on one of the sites as well and will report how this workaround went.

heikkiy’s picture

Hi @yevko, nice to see you.

No, both sites have the default language as English.

Our system.site.yml has

_core:
  default_config_hash: yXadRE77Va-G6dxhd2kPYapAvbnSvTF6hO4oXiOEynI
langcode: en
uuid: 01602f5c-032b-4c67-a1da-b7bba1207a45
name: 'Example site'
mail: 'noreply@example.site'
slogan: ''
page:
  403: ''
  404: ''
  front: /node/1
admin_compact_mode: false
weight_select_max: 100
default_langcode: en
mail_notification: noreply@example.site

But indeed the language negotiation has been set so that the site by default opens the Finnish site and not the English one in language.negotiation.yml:

_core:
  default_config_hash: uEePITI9tV6WqzmsTb7MfPCi5yPWXSxAN1xeLcYFQbM
session:
  parameter: language
url:
  source: path_prefix
  prefixes:
    en: en
    fi: fi
  domains:
    en: ''
    fi: ''
selected_langcode: fi

My feeling is that because the site's default language is set to English sometimes Drush takes that as the default language and changes Finnish labels into English. This does not happen the other way around that English site shows Finnish labels. The problem still gets fixed by clearing all caches in the admin UI.

joseph.olstad’s picture

@heikkiy , we have noticed exactly what you described.

Someone else on my team fixed this and I'm looking for which patch they used.

Turns out we're using this one https://www.drupal.org/files/issues/2024-02-23/3221375_lost-field-labels...
D10.2.5

patch #50

Thanks!

joseph.olstad’s picture

**EDIT**
nvm
**EDIT**

tstoeckler’s picture

So I hit this as well now, yay. Similarly to what is described above we use the Selected language negotiation plugin to have a current language that is different from the default language and then the field labels incorrectly end up being in the default language instead of the default language after a drush cr.

I read through the above and have the following thoughts:

  1. Various related things have been discussed above (in particular browser language negotiation in #14-18 and taxonomy term labels #32 and #46), but I think it makes sense to focus on the field definition cache problem for this issue as that is what most people have reported and that is a very concrete cache poisoning problem, in particular because we explicitly add the language code to the cache ID. So there's no arguing that storing the German word "Kopfzeile" as part of a definition with the cache ID entity_base_field_definitions:node:en (instead of entity_base_field_definitions:node:de) is not a bug, regardless of any other issues related to language negotation. Agreed that there are various structural and systemic problems in the whole language subsystem that are being touched here, but we have a concrete bug here, so let's use this issue to fix that bug.
  2. (Speaking of more fundamental problems:) Agreed that ideally there would be some sort of "init" mechanism for Drush to call where language module would do the language negotiation similar to LanguageRequestSubscriber. But, alas, we don't have that, and introducing that is not even remotely a trivial thing to do, so let's just ignore that for the purposes of this issue. (Just wanted to note explicitly that I totally agree with the various assessments to this end raised above.)
  3. Focusing more on the field label issue at hand now, and thinking about the involved subsystems I originally came to the conclusion that we could maybe fix this directly in ConfigEntityStorage which is where we transition from the config subsystem to the entity subsystem. This is where the - potentially overridden - config values are used as values for entities which do not have the notion of overrides (or translations, in particular). So putting it that way I thought it makes sense to at that point, i.e. in ConfigEntityStorage::doLoadMultiple(), make sure that the current language config overrides are loaded. The problem is that with the explicit setConfigOverrideLanguage() API there is no way to detect if a language has been set explicitly or if it is just the fallback. So if I do
    \Drupal::languageManager()->setConfigOverrideLanguage(Language::load('en'));
    FieldConfig::loadByName('node', 'article', 'heading')->label()
    

    That needs to return the English string "Heading", regardless of the default language or the language negotiation configuration. But inside of ConfigEntityStorage::doLoadMultiple() there is no way to tell if English was set as the override language explicitly as above or as a fallback because it is the default language. So as far as I can tell, it is not possible to fix the issue at that level, even though it could be considered the correct place to do it

  4. I think the next best place to fix this is at the place it is causing problems. The current merge request adds the overrides in EntityFieldManager which is fair enough. One could argue though that the field manager should not know about the particulars of config overrides and in fact, the concrete problem only arises with field module's usage of field definitions (as they are config-backed), which are loaded in \Drupal\field\Hook\FieldHooks::entityBundleFieldInfo(). However, the current merge request does not take into account base field overrides which may suffer from the same problem. These are loaded directly in EntityFieldManager::buildBundleFieldDefinitions() so we have to load the overrides there anyway.
  5. As #41 points out we should revert the config override language back afterwards, in case it was set explicitly. Taking the previous point into account I think it makes sense to do things in the following order: 1. load config overrides, 2. load base field overrides 3. load bundle fields from hooks (which loads config fields) 4. "unload" config overrides
  6. Theoretically field storage definitions suffer from the same issue, but they do not have any translatable data. Theoretically a field could provide a translatable storage setting, but this somehow goes against the whole point of storage settings as the value of a storage setting should not change depending on the current language. So I would not change anything related to storage definitions but add a comment to that effect in case someone does hit that particular problem down the road.
  7. As #46 points out LanguageRequestSubscriber does two other things than language negotiation: 1. Setting the current user on the negotiator and 2. setting the default language for string translation. For 1. I am not sure why that happens in the request subscriber vs. just injecting the current user service, maybe just for historic reasons, i.e. at the time the current user service didn't exist in its current form. Either way I will open a separate issue to look into cleaning that up, so that we don't have to deal with that here. For 2. I don't think we have to deal with string translation when just dealing with this particular issue, that would only come into play with a more generic "init" mechanism for Drush, as far as I can tell

So having said all that I will now open separate issue for the current user thing mentioned above and try to write a failing test with a language override for a config field and a base field override and then make the test pass.

tstoeckler’s picture

Status: Needs work » Needs review
Issue tags: -Needs tests

Opened #3544395: Current user missing from language negotiator outside of request context for the other issue. That ended up being a bit more "interesting" than anticipated, that actually revealed that language negotiation is not even run at all in CLI requests, because the current user is never set on the language negotiator. So people experiencing the incorrect field labels will need the patch from over there, as well. (I still think it makes sense to keep these separate, as in my opinion they are separate (albeit related) fixes, but your mileage may vary.)

Then as per the above added a test for a language override of a field config and and base field override and verified that the proposed fix does indeed fix that issue.

tstoeckler’s picture

Here's the test only jobs with the two failures in the newly added tests. https://git.drupalcode.org/issue/drupal-3221375/-/jobs/6408136

smustgrave’s picture

Status: Needs review » Needs work
Issue tags: +Needs issue summary update

Not sure which MR is up for review can the issue summary be updated with this. If both are 2 approaches lets include in the summary optionA and optionB

smustgrave’s picture

Hiding patches.

liam morland’s picture

Merge request 13139 is for Drupal 11. The other is for Drupal 9.

tstoeckler’s picture

Issue summary: View changes
Status: Needs work » Needs review
Issue tags: -Needs issue summary update
Related issues: +#3538006: Optimize EntityFieldManager::buildBundleFieldDefinitions()

Fixed the conflicts after #3538006: Optimize EntityFieldManager::buildBundleFieldDefinitions() and updated the test to pass again.

Also updated the issue summary with the proposed resolution.

geek-merlin’s picture

Status: Needs review » Needs work

@stoeckler Thoroughly debugged, thx a lot! I especially like the explicit reasoning.
And here is where i disagree on a small but crucial assumtion:
> (3) The problem is that with the explicit setConfigOverrideLanguage() API there is no way to detect if a language has been set explicitly or if it is just the fallback.

In LanguageConfigFactoryOverride we CAN negotiate the configOverrideLanguage on demand instead of relying on the explicitly set value, and ONLY fall back to default language on recursion. Some while ago i implemented this in a project and never found time to upstream it. I will give this a MR shot soon-ish.

tstoeckler’s picture

Status: Needs work » Needs review

Not exactly sure what you mean by "on-demand". Are you proposing to remove the setting of $this->language in LanguageConfigFactoryOverride::__construct()? It seems to me - regardless of the merits of that proposal - that that is a larger change, independent of this issue.

As far as I can tell, you did not find anything actually wrong with the MR, so even if you do have the idea for an improvement, let's keep this at needs review until that improvement actually arrives.

geek-merlin changed the visibility of the branch 11.x to hidden.

needs-review-queue-bot’s picture

Status: Needs review » Needs work
StatusFileSize
new2.86 KB

The Needs Review Queue Bot tested this issue. It fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.

hlopes’s picture

I'm seeing something similar here, here's what i got:

So we have some plugins + config translations for those plugins.

When clearing up the cache, if in a different language, the plugin definition comes with the translation overrides and gets cached in cache_discovery table. Further instantiations of this plugin will use this cached version, regardless of language, leading to labels and what not in the wrong language.

When the config translations are deleted, existing translations in locale (?) (/admin/config/regional/translate) revert to the current english (?) value.

Reimporting core translations re-adds those, so after doing so everything works as expected. Plugin is loaded in the only language it knows (english), and translated later in the pipe (in our case, in a form).

kmonty changed the visibility of the branch 3221375-wrong-language-field to hidden.

kmonty’s picture

Hid the old, out-of-date D9.3 branch (per feedback in #63 and #64).

Issue summary still needs updates between the main MR !13139 (which should be NR) and @geek-merlin's !14212 fork (which is NW).

kovhan’s picture

We have a Drupal site where English is the Drupal default language, but Swedish is the primary language — all content, UI and field labels are configured in Swedish.

When the search index rebuild runs as a batch process, field labels get cached with English text instead of Swedish.

Root cause:

During batch indexing, configOverrideLanguage is misaligned with getCurrentLanguage(). EntityFieldManager::buildBundleFieldDefinitions() reads field config in English (the default) and caches the result, poisoning the cache for Swedish users.

Fix:

MR !13139 resolves this completely by saving and restoring configOverrideLanguage inside buildBundleFieldDefinitions(), ensuring field labels are always loaded in the correct language.

Drupal 11.2.5

kmonty changed the visibility of the branch 3221375-language-override-fix to hidden.

kmonty’s picture

Status: Needs work » Needs review

Hiding the alternate PR that is a WIP. Marking the active PR as Needs Review.

needs-review-queue-bot’s picture

Status: Needs review » Needs work
StatusFileSize
new549 bytes

The Needs Review Queue Bot tested this issue. It fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

smustgrave’s picture

Status: Needs work » Needs review

Bot rebellion

tstoeckler’s picture

Fixed the PHPStan failure, presumably due to stricter checking that was enabled in the meantime. There is currently some hickup with the Javascript tests, which is why the pipeline fails, but as far as I can tell, there is nothing that is actually caused by this issue.