Problem/Motivation

As you create configuration on a Drupal site, it may be created in one of 4 languages:

  1. Current request language
    1. Configuration created on the UI that does not have a language selector (eg. content types, block types, etc.). This means if an admin browses the site in a foreign language, if they create a new content type, it will be created in that foreign language.
    2. Configuration created through config actions (eg. in recipes). This means that users of project browser will have configuration created in recipes with config actions created in the language they happened to use in their browser.
  2. Language explicitly selected on the UI
    1. Very few config entities have an explicit language selector (in core: menus, views, taxonomy vocabularies, date formats). This means that even if your request language is X, you can still create that config in Y, Z, etc. languages using the selector. This is the only way to create in "Not specified" and "Not applicable", which are two options offered.
  3. Configuration language directly in a YAML file, even if the language does not exist on the site at the time(!) (or if missing assumed as English)
    1. Configuration shipped modules, when the module is installed if locale module is not yet installed, regardless of the site default language!. This means that if your site default is German but locale module is not installed, module config will still be imported in English. Even if the English language was deleted earlier.
    2. Configuration shipped in recipes, when the recipe is applied regardless of the site default language or if locale is installed or not!. Even if locale module is installed recipe shipped config is always installed in the language it is in, regardless of whether that language is configured on the site or not.
  4. Site default language
    1. Configuration created through extension installation if locale module was previously installed already on the site. When a module is installed, its config is created in the site default language at the time.

So you already have config in various languages. Then Drupal tries to keep the config language up to date but that only ever happens with extension installed config. But!

  1. Extension config language update only happens when extensions get installed. If you modify the site default language, you need to THEN install an extension for the config of all previous extensions too to get updated.
  2. The update only ever happens if the config was still English and was now switched from English to something else. If it was not English, than a site default language modification does not result in config language modification even later when extensions get installed.
  3. Once again this modification only applies to extension installed config, not manually created config or recipe installed config or config action created config.

See locale_modules_installed() and core/modules/locale/src/LocaleConfigManager.php for this logic.

Steps to reproduce

  1. Install Drupal standard in English
  2. Enable the language module and add German on admin/config/regional/language.
  3. Create a new content type. This will be English. Existing installed config will be English.
  4. Create a new menu. Note that you can pick English or German for this menu. Create it in German. Config export will show the config entity got created as German.
  5. Go back to create a new content type. Add a language prefix (/de for German) to the URL and load that page. While it is not apparent in this UI, this will create the content type in German. Verify with config export.
  6. Add another language (for example French) on admin/config/regional/language
  7. Change the default language to French that you just added.
  8. Install Media module. The installed config will be English even though your site default is now French.

Another variant of the steps to use interface translation from the start for different dramatic effect:

  1. Install Drupal standard in English
  2. Enable the language module and Interface translation modules. Add French on admin/config/regional/language.
  3. Create a content type. This will be English. Verify with config export.
  4. Change the default language to French that you just added.
  5. Create a content type. This will be French as the URL has /fr in it because modifying the site default to French brought you there. Verify with config export.
  6. Note that all of your config other than the French menu is still English. Verify with config export.
  7. Now install Media module. A big config update will happen that updates extension installed config to be French. Your English content type remains English. Verify with config export.
  8. Now switch the site default language to German. Nothing happens to config. Verify with config export.
  9. Install search module. This will still NOT rewrite your existing config to be German but search module related config will be installed in German now. Verify with config export.
  10. Now you have English, French and German config, most config is French.

Also, if you then switch site default language back to English, the configuration language for existing config also never switches back, not even when you install more modules. (This is true at any point after the site default language was once a foreign language).

Also, if you delete either of the 3 languages, which can be done without issue, you are left with configuration in that language Drupal does not know about.

Proposed resolution

The site's configured default language (system.site:default_langcode) is the single source of truth for what langcode new or existing configuration without a langcode gets. A new event lets every installed subsystem contribute to that question, with the most specific answer winning.

The ConfigLangcodeResolveEvent event

Drupal\Core\Config\ConfigLangcodeResolveEvent is dispatched to determine a configuration's langcode. It is only dispatched when the system determines it is necessary. This depends on whether the configuration has a langcode already or if config is being created by the ConfigInstaller. It carries no information about the object being saved. A subscriber calls setLangcode(). The first call wins and stops propagation, so a higher-priority, more specific subscriber takes precedence over a more general one, without inspecting what any other subscriber decided. This allows contrib or custom code to completely override what Drupal core does. getLangcode() never returns null. Core always registers a subscriber that answers unconditionally.

Dispatched by:

  • Config::save(), for simple configuration that is translatable and has no langcode.
  • ConfigEntityStorage::doCreate(), for an entity created without an explicit langcode.
  • ConfigInstaller::createConfiguration(), once per call to createConfiguration() for the default collection, for extension-shipped configuration with 'en' or no langcode.

How this works for Drupal core

During site installation: ConfigLangcodeInstallerSubscriber responds with the language chosen for that installation - at the highest priority, so shipped configuration installs in the chosen language before system.site itself has been set.

After site installation: ConfigLangcodeSiteDefaultSubscriber resolves to system.site:default_langcode. Unless the language module has been installed this will be 'en'. If the language module has been installed it will be whatever the user has selected as the site default language.

ConfigInstaller::createConfiguration() treats a missing langcode as English by default and resolves it through the same event. An explicit, non-English langcode is left alone. Recipes install configuration through the same code path (RecipeConfigInstaller extends ConfigInstaller), so this applies equally to recipe-shipped configuration.

What happens when the site's default langcode is changed

This can only occur if the language module is installed. The language module triggers a batch controlled by the Drupal\language\DefaultLangcodeConfigUpdater service. It updates the default configuration langcodes and preserves translations in config overrides for the new langcode before it removes them. It also adds config override for the old default language.

If locale is installed, it does two things during the default langcode change. First, LocaleConfigSubscriber ignores config and override saves while DefaultLangcodeConfigUpdater::isReconciling() returns TRUE. This stops those saves from being stored in locale as customized translations. Second, locale subscribes to DefaultLangcodeConfigReconcileEvent. The event is dispatched before each config object is saved. Locale translates the data into the new default language. Values translated by the old default, but not by the new default, go back to the English source. The config is saved once. So installing German and then adding French and making it the default language gives the same result as installing French and then adding German.

Previous solution

We tried to address this problem by adding a new setting for a default config langcode but this hit snags because we still had to deal with the site's default langcode changing. See the hidden MR for how this approach worked https://git.drupalcode.org/project/drupal/-/merge_requests/16280

Remaining tasks

  • Code review
  • Create any necessary change records

User interface changes

New batch triggered when changing the default langcode

API changes

  • New: ConfigLangcodeResolveEvent event. It is dispatched when a config object's language is unknown or when it is English and the config is being created by the ConfigInstaller.
  • New: Drupal\language\DefaultLangcodeConfigUpdater service - creates a batch to update config when the default langcode changes. This uses the new interface Drupal\language\DefaultLangcodeConfigUpdaterInterface
  • New: Drupal\language\Event\DefaultLangcodeConfigReconcileEvent. It is dispatched before each config object is saved. Subscribers can change the data with ::setData() or skip the config with ::skip().
  • New: Drupal\locale\LocaleConfigManager::getDataForDefaultLangcodeChange(). Returns config data for the new default langcode, without saving. Keys still in the old default language fall back to English. Locale translations for the new langcode are applied on top.
  • Changed: LocaleConfigSubscriber::updateLocaleStorage() and ::processTranslatableData() each take a new $original_config parameter.
  • Changed: LocaleConfigSubscriber::__construct() takes new $configInstaller and $defaultLangcodeConfigUpdater parameters. Calling it without them is deprecated.
  • Deprecated, no replacement: LocaleConfigManager::updateDefaultConfigLangcodes().
  • Deprecated, no replacement: LocaleConfigBatch::batchUpdateDefaultConfigLangcodes() and the $update_default_config_langcodes parameter of LocaleConfigBatch::buildBatch().

Data model changes

  • Config::save() adds a langcode key when config becomes translatable and none is present, and removes langcode when config stops being translatable.
  • system_post_update_reconcile_translatable_config_langcode is a new update that ensures simple configuration has a langcode if it is translatable and does not have a langcode if it is not.

AI Usage

alexpott used AI to review code, propose solutions, write tests, implement ideas and help write the issue summary. alexpott made every commit, and reviewed and tested each change before committing it. Design decisions are alexpott's.

Release notes snippet

Configuration installed by modules, themes and recipes now gets the site's default language as its langcode. This happens with or without the Interface Translation module. Configuration entities created without an explicit language, for example a new content type, also get the site's default language instead of the current interface language.

When the site's default language is changed on the language overview page, a batch updates all configuration in the old default language to the new default language. The previous values are stored as translations in the old language.

This batch does not run when the default language is changed outside a form, for example with Drush.

Issue fork drupal-3337864

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

alexpott created an issue. See original summary.

tsotoodeh’s picture

The same issue observed in 9.5.x, it is a core design issue, I opened a multilingual support issue:
Multilingual support of strings in second language website

I Tried using configuration update manager module to adjust langcode from default language to primary language, time consuming process! Although the content is translated but it would not shown correctly in second language. Same issue here!

jose reyero’s picture

Issue summary: View changes
jose reyero’s picture

Updated task description to account for the fact that when you switch default language back to English, nothing happens, existing config language is never updated.

I guess we need to make locale_config_batch_set_config_langcodes() behave the same for all languages and not to make an exception if default language is "en", or maybe keep track of the last language set for configuration, and whether the default langcode has been changed or not...

tsotoodeh’s picture

Well, as far as I can see the Drupal multilingual efforts has come a long way from the time that i18n or internationalization and localization modules was once were.
To be truly multilingual the configuration translation and interface translation should be taken a second thought.

Updated task description to account for the fact that when you switch default language back to English, nothing happens, existing config language is never updated.

As you said, the the language switch should be respected both in terms of configuration of the elements of the site and also the contents. An architectural plan should be devised for this task. Currently the configuration in consistency after switching the language from default language are many to account.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

almador’s picture

I don't know if this is the right topic to share my issue, but it's also related to switching the original language.

What I did - while upgrading Drupal 8 to Drupal 9 by mistake I changed my default language from English to Spanish. When I realized this (after the upgrade) I switched my default language back to English from this page:
admin/config/regional/language

But now, when I'm trying to edit any of my translations related things it still says "Español (original)".

Is there any way to switch every translation option back to English (original)?

gábor hojtsy’s picture

Title: Changing the site default language after install results in complex breakages » Configuration language is not adjusted after changing the site default language

Made the title more specific.

gábor hojtsy’s picture

Looking at why this happens. When a module is installed after the default language is changed:

  • locale_modules_installed() is called (this is a hook that responds to modules being installed).
  • This calls locale_system_update() which sets up a batch.
  • The batch first runs translation updates (downloads .po files from localize.drupal.org and imports them). This is not relevant for this bug.
  • Then it adds a sub-batch with locale_config_batch_update_components() passing the request to update default langcodes in config. Because there are no specific components requested, this will get a list of all the default config objects (config items that are shipped with installed extensions). See https://api.drupal.org/api/drupal/core%21modules%21locale%21src%21Locale...
  • Then it invokes locale_config_batch_build() which uses this component list of all default config and adds two kinds of operations: (1) to update the langcode of them and (2) to update the translations of them.
  • The batch operation locale_config_batch_set_config_langcodes() just wraps updateDefaultConfigLangcodes(). This operates if the site default langcode is not English and updates all of the active copies of default config that do have an English langcode or are missing a langcode to the current site default langcode. This does not change existing active config if it is already some other language. It also does not change the active config back to English if the new default is English. It just does not operate in that scenario.
  • Then a batch operation is added for 20 config names each of locale_config_batch_refresh_name(). This wraps updateConfigTranslations(). This considers the active config language and detects if the translation should be stored within the active config object or an override. This is independent of the site default language and only considers the langcode of the active config object.

All in all I think the logic protects the existing language changes in active config if they were made prior to the default language change. So if you already have a Swedish site let's say, you switch to Finnish, the active config items that were previously Swedish will remain Swedish. Newly installed modules will get their config in Finnish in active config, so you end up with a mix of config items with Swedish and Finnish. But the integrity of your config is maintained.

If you built out a part of the website in English and then switch the default langcode, this indeed ends up doing more dramatic changes later because English is considered special. (When the default language is set back to English, I think the data protection logic does what it was supposed to do).

Now as to what happens when default language is changed, I'll post another comment, this is already quite long :D

gábor hojtsy’s picture

Default language change on the UI is implemented in LanguageListBuilder::submitForm(), and does this essentially:

   // Save the default language if changed.
    $new_id = $form_state->getValue('site_default_language');
    if ($new_id != $this->languageManager->getDefaultLanguage()->getId()) {
      $this->configFactory->getEditable('system.site')->set('default_langcode', $new_id)->save();
      $this->languageManager->reset();
    }

    if ($this->languageManager instanceof ConfigurableLanguageManagerInterface) {
      $this->languageManager->updateLockedLanguageWeights();
    }

Language module reacts on the config save of the default langcode in its ConfigSubscriber but that of course is not related to interface translation.

Finally locale has its own LocaleConfigSubscriber but that does not consider a default language code change as something to care about.

gábor hojtsy’s picture

I'm a bit torn as to how to resolve this. For one, we want to make default language changing as easy as possible. The intent of the code is to consider config similar to content, where the existing language and overrides of configs is kept, except in the English special case that it assumes belongs to the default config that needs changing. There is no such logic for content entities either to swap their default language and translations when a site default language changes.

At the same time, users likely expect to edit configuration in the site's default language by default, which the current logic attempts to ensure, at least as long as that default is English and after the first transition from English to non-English. It does not maintain that user experience once you want to make the second language transition, whether back to English or to something else. It does leave your config intact but it does not maintain the assumed user experience. Also, if the first language transition happens later in the site's life, a massive config change happens then. That in itself is expected, but not when you install a new module :D So the problems are:

  • When you change the site default language, the current logic is not run. That would probably be "easy" to fix by launching the existing batch then, since it deals with all default config. Not sure how to make it happen in a config import though?
  • After you change the site default language AGAIN, the current logic keeps your config as-is (not English) and new config is now created in the new default language. To maintain the idea that your active config is in the site default by default, we would need the logic explained in the issue summary to extract the translations from active config, create the overrides and save the active config with the proper values fom the overrides in the new default (and remove the new default's overrides for default config).
drupaldope’s picture

It seems that someone has been changing some things in how languages work.

I am now unable to rename a certain language (D 10.2.4).
I wanted to name English en and German de.
The YML configuration is correct, but when the page is shown in the default original language German, it will just say "Deutsch" no matter what.
I guess the new code changes fail to retrieve and display the correct label.

drupaldope’s picture

Actually I had it working correctly on a site where I did the following :

Installed the site in English

Added German
Added British English
Deleted English

works.

drupaldope’s picture

there is actually more to this issue.

languages need a translatable label, which would resolve the present issue and a dozen more related issues I have found today when going down this rabbit hole.

BUT
languages also need editable labels that will be displayed in translations, verbose translations and abbreviated translations, URLs ...

So for example, a site could have installed British English and it would display the following way :
"en-gb" (non translatable) in any technical-related stuff such as hreflang, html, etc.
"British English" (translatable) in any configuration dialog in the admin interface, translation forms, etc.
"en" (translatable) as a label for language switcher links
and potentially, the site owner would want something else (translatable) to be displayed in the URL, such as /eng/ or /english/ or ... whatever

so the short ISO language code would be used by Drupal to identify the language and source translations, and would be untranslatable.
and then there should be 3 translatable labels on top.

drupaldope’s picture

I can now also confirm the workaround to correct this issue on running sites.

First, install an additional language.
Then change the language of all localized content to that new language you just installed.
That may be nodes, blocks, media, redirects and URLs, etc.

Then delete the language of which you can't change the name (be careful, this will delete all content you forgot to switch to the new language).

Purge the cache and then re-install the language you just deleted.

Edit the name of that language to what you wanted and then change the contents' language back to what it was.

So it seems like something screws up the language settings when installing the site or when declaring a language default.

drupaldope’s picture

never mind the workaround, one of the language labels reverted back to original after doing something else on the site.

claudiu.cristea’s picture

I don't know whether is related to this. I set the default language to other than English. Then I've exported the config (drush cex). Reinstalled the site from config but the English language is gone. Checked the config sync directory, I can see the language.entity.en.yml file. Checked the {config} table, no language.entity.en at all. It just vanished.

EDIT: Ignore this comment. I just discovered that I need to set keep_english: true in the installed profile. Maybe this deserves to be better documented, even by showing a warning message to the site builder when they are changing the default language?

EDIT2: I have created #3545306: Installation from config removes English if the installed profile does not have keep_english for this issue

anybody’s picture

Nice, just found this and I think it's very similar to this one: #3568743: Convert / unify all default configs to the site default language (which has a possible solution).

For the future I'd vote to always store the default config in English by design, so we can't have such cases any more and treat any other language as translations in their respective collection / folder!

That will also simplify importing (typically English) config from modules etc...

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.

dqd’s picture

Added to the "Refine settings, flexibility and coherences between translatable areas" outline on git.drupalcode.org. Thoughts appreciated.

ressa’s picture

Connecting related issue and adding "Update Install a language doc page" in Issue Summary as a task.

gábor hojtsy’s picture

@anybody: in some European studies (eg a study of 100k websites of in the EU, 4-5k sites per country) it was found that 80% of the sites are not English single language. Why would they need to create content types, views, etc. in English? That would be bad :) Drupal already takes some effort to keep the default config in your language, we need to fix the missing use cases.

I wrote out some plans above 2 years ago in #9, #10 and #11. Since then I realized we probably don't need to deal with this in config import as that would assume you are already deploying an updated set of config. So we need to do this on the form. Since that already happens later when you install a module, we should at first bring the same behaviour forward to the default language change IMHO as a first step.

anybody’s picture

@gábor hojtsy thanks. Yes I understand that. My recurring issue in the role of a site builder is, that again and again I have to think about in which language I'm acting and how I should name even the machine names.

In 60% of our cases the projects are in German, some of them are ONLY in German and some of them are translatable into other languages. But we love the configs to be consistently in English as base language..

From our perspective it would be more appropriate to be able to rely on English as base language and use translation for anything else.
You know a lot more about the internal concepts and I think you discussed this 100000x more than me and have good reasons. So this comment is not meant to tell what should be done, just to let you know about the background and our daily pain with the "mix" we're experiencing.

Still my solution would not save us from having clients that then create entity types with German names ;) So there's no perfect. Most importantly the bug needs to be fixed, as you wrote. For the other things we'll still have to find a better / best practice.

Thank you so much! :)

gábor hojtsy’s picture

Assigned: Unassigned » gábor hojtsy

I'm working on implementing this now :)

gábor hojtsy’s picture

@andybody: for your base English use case you can install the site in English and keep site default in English, why not do that?

gábor hojtsy’s picture

Status: Active » Postponed

While I pushed a complete lifecycle solution above in an MR, I decided that this will not be the best route :D Instead I think there is a lot of merit in trying to cater more for what @anybody is saying about consistent source language AND decouple the config default language from site default after that (so changing config default language is a conscious choice separate from site default). For the first part of this, I opened #3608533: Introduce "Configuration language" settings page to control langcode visibility. Once that lands, I think we can return here and introduce the Config default language and adjust the code to use that when dealing with config.

I'll work on #3608533: Introduce "Configuration language" settings page to control langcode visibility tomorrow, marking this postponed on that. I already posted the rationale and high level plan there though :)

anybody’s picture

@gábor hojtsy re #25 we do that in many cases, but it doesn't feel perfect because then you have the English language also in many Content (related) areas. It just doesn't feel natural and correct to me. And you still have to be very careful to not accidentally use the wrong language somewhere.

#27 sounds like you came to a similar conclusion. Thank you so much for all your work put into this.

gábor hojtsy’s picture

Title: Configuration language is not adjusted after changing the site default language » Introduce a dedicated "Configuration default language" different from "Site default language"
Issue tags: +Needs issue summary update

@anybody: yeah I did came to the conclusion that we need to decouple the "Configuration default language" from the "Site default language", which is another way to approach "enforce a config language". I also came to the conclusion that the few core changeable config languages are in the way sort of :D So I created #3608533: Introduce "Configuration language" settings page to control langcode visibility and proposed a solution there. That introduces explicit intent about language of those config, so after that as we want to I think introduce a dedicated "Configuration default language" here, we can rely on that intent / configuration to not change things that site owner explicitly wanted to be in other languages but change the rest of the config as needed. (Also to not change config language on site default language change ever). Let's reposition this issue for that, but we need to resolve #3608533: Introduce "Configuration language" settings page to control langcode visibility so we have data on the intent of the config that is currently potentially in other languages. That also helps enforce a consistent language if you don't configure those entities to be able to pick a language which is the new core default as proposed there.

Please review that one to push this cause forward :)

gábor hojtsy’s picture

Created a parent issue to group related problems and show how this leads to resolving config language issues once and for all hopefully :D

gábor hojtsy’s picture

Issue summary: View changes
Issue tags: -Needs issue summary update

Rewrote the issue summary. Will of course need more expansion but this is a good start IMHO.

gábor hojtsy’s picture

Issue summary: View changes

Fix table HTML.

gábor hojtsy’s picture

Issue summary: View changes

Fix one more HTML issue and reorder the events in the HTML to make it easier to follow.

gábor hojtsy’s picture

Issue summary: View changes

More elaborate items in the table to point out the problems better.

gábor hojtsy’s picture

Issue summary: View changes

Fix list markup.

gábor hojtsy’s picture

Issue summary: View changes
gábor hojtsy’s picture

Issue summary: View changes
gábor hojtsy’s picture

Issue summary: View changes

Massive update to issue summary based on recent discoveries. I found 3 new problems since the above, so getting more convinced that we want to have one unified "Configuration default language".

gábor hojtsy’s picture

Issue summary: View changes

Clarified a few items.

gábor hojtsy changed the visibility of the branch 3337864-configuration-language-is to hidden.

gábor hojtsy’s picture

Status: Postponed » Needs work
roderik’s picture

Thank you for this :-)

I was tracing a situation that I wasn't sure is 'my' bug or not, and I haven't seen it mentioned anywhere:

The samlauth module starts out with very minimal installed config, with no label values and no language code. But there are labels in the config object, and as soon as you set a label value... Configuration Inspector reports a missing language code. #3611776: samlauth.authentication config object must specify a language code

It felt like either I was missing how exactly I should solve this... or the only way to solve that reliably would be adding code into Config::save(). (Which felt like too big a thing to propose for someone without intricate knowledge of the config system...)

Luckily, I read your latest MR's code and it does just that! So... I believe I bumped into yet another issue that is solved by your MR!

gábor hojtsy’s picture

@roderik: I have a targeted issue for that problem at #3600904: Shipped configuration entities that do not have translatable elements are not set to the site default langcode as that is also reproducible with core only :)

gábor hojtsy’s picture

I've been experimenting how far this needs to be stretched in the past few days and ended up building https://www.drupal.org/project/config_language_lock which has about 900 lines of PHP code and almost 2x the amount of that in docs and 4x the amount that in tests to prove how core works vs how the module works. People following this issue would be welcome to test that (in dev environments for now, it is very early stages). That said the docs and tests and SUPER thorough, so I'm fairly confident it is fine. Read the very extensive docs at https://project.pages.drupalcode.org/config_language_lock/

roderik’s picture

D'oh! I failed to read this issue's meta parent and its children.

Reviewed #3600904: Shipped configuration entities that do not have translatable elements are not set to the site default langcode. The issue I stumbled into is not currently solved by it though (because it's simple config), so curious for the next step.

alexpott’s picture

Status: Needs work » Needs review

I changed the MR to result in less changes by ensuring we're only adding a langcode to config with translatable parts. I've also enforced the 'en' default in code which makes the change less disruptive to kernel testing where the system module might not be installed.

catch’s picture

It would be useful for installer refactoring if we were able to move the default language configuration out of system module to core/

This is because if you try to install system module along with any other modules at the same time, it's impossible to set the system default language in between system module being installed and the other modules getting installed, and that information comes from the request.

Currently working around this in #3614153: Install system module alongside other modules in the installer where you can see having the config outside system would make it a lot easier.

alexpott’s picture

@catch sure we can introduce a core.language and open a follow-up to move the site default langcode there too. Sometimes I wish we could move this to container parameters instead of config.

We also need to prevent the language that is the default config langcode from being deleted.

needs-review-queue-bot’s picture

Status: Needs review » Needs work
StatusFileSize
new91 bytes

The Needs Review Queue Bot tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".

This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

alexpott’s picture

I spoke with @Gábor Hojtsy about this issue. Two things came out of that conversation:

1. A new blocking issue is needed:

  • We need a new (blocking) issue to give the typed config manager the ability to determine whether a config object is translatable. The current approach, inferring this from data + schema, produces incorrect results. More detail on #3616515: Add \Drupal\Core\Config\TypedConfigManager::hasTranslatableValues()
  • Once that's resolved, we'll come back to this issue and use it to determine when to add the langcode to configuration.

2. Test coverage and scope for this issue:

  • The current approach here will break creating a view in English on a multilingual site where English is enabled but is not the default language. We need to add test coverage for this scenario, and for other language-configuration variations, to this issue.
  • We also need to be more careful about changing langcode when configuration is created — this should really only happen at install (extension, site and recipe) time, not on every save, and we need to consider what should occur if the langcode of the configuration is supported (or not) by the site.
gábor hojtsy’s picture

Other than "changing langcode", there is also applying langcode for config actions (can happen through ECA for example too?) and UI creation of config, which should use the same config language by default. But not general write of all of config as we agreed to keep the existing flexibility if people really want to use it.

alexpott’s picture

So I worked on improving the current approach - moving away form core.language as we only need the config when the language module was installed and introducing an event architecture to resolve the config langcode - this allows us to do different things when the installer is in play or the language module and also allows us to know when we;re resolving a langcode during an extension install and when it is regular runtime. I also added an exception when you delete the language that is the config default. And this is where I hit a big block - we have a number of tests that expect to be able to change a site default language and delete the old one - for example \Drupal\Tests\config\Functional\ConfigLanguageOverrideWebTest::testSiteNameTranslation.

This has led me to the conclusion that we need to allow the config default langcode to be changed. And once we think this then we actually do not need to introduce a dedicated configuration default language but we should use the site default langcode and support (and test extensively changing it). I've done this and in my next comment I'll detail all the changes to support this. It builds off much of the work done on this issue and think we should re-frame this issue to do take this approach. I've updated the existing MR https://git.drupalcode.org/project/drupal/-/merge_requests/16280 0 will all my work to get to this point so if we pivot again back to a separate config default langcode setting then we're nearer completing.

alexpott’s picture

I've created a new MR https://git.drupalcode.org/project/drupal/-/merge_requests/16913 that does not introduce a new config default langcode - but rather uses the system.site default_langcode and fixes and tests the behaviours around changing it.

Here's a summary of the changes:

  • Introduces ConfigEvents::RESOLVE_LANGCODE, replacing ad hoc default-langcode logic across Config::save(), ConfigInstaller, and config entity creation.
  • Config::save() only adds or removes a langcode when a config object's translatability changes. It never overwrites an existing langcode. This fixes explicit English configuration being rewritten to the site default on every save. Config entities are always considered translatable.
  • system_install() now sets system.site:default_langcode from the language chosen during installation.
  • ConfigInstaller relabels extension-shipped configuration with no langcode, or en, to the site's default. It rejects configuration shipped with an unsupported explicit langcode. Without language module installed, only English is supported.
  • Changing the site's default language retroactively relabels existing configuration from the old default to the new one, and locale module retranslates it. The old default's content is preserved as a configuration override. An existing override for the new default is merged into the active configuration and removed.
  • Calling LocaleConfigManager::updateDefaultConfigLangcodes() or its batch operation now triggers a deprecation notice; there is no replacement.
  • A post_update hook reconciles existing sites' configuration langcodes.
alexpott’s picture

Status: Needs work » Needs review
alexpott’s picture

@Gábor Hojtsy pointed me towards https://git.drupalcode.org/project/config_language_lock/-/work_items/361... and initially I thought that this puts a big hole in my new approach in #55 but after working with Claude on the issue I've realised that actually if change the ConfigEvents::RESOLVE_LANGCODE event so that only the first resolved langcode wins we actually provide a pretty easy way for https://www.drupal.org/project/config_language_lock to support this exact use-case as long as we file a follow-up to improve config entity editting when the site is multi-lingual.

svendecabooter’s picture

We have a site in a messed up configuration-language state, which this issue is aiming to resolve.
I'm aiming for the following setup: site default language Dutch, English enabled but not the default, and all configuration authored and shipped as langcode: en.

Since we use recipes for adding extra features regularly, I tested this MR against that scenario to see how a recipe carrying en config behaves when the site language is nl.

The result is that a recipe delivers configuration by two routes that now disagree with each other. Setting the site default to nl and applying a recipe that (a) ships langcode: en config in its own config/ directory and (b) installs a module shipping langcode: en config:

- the recipe's own config/ file resolves to nl - correct
- the config shipped by the module the recipe installed stays en
- the same module installed directly, not through a recipe, resolves to nl

So the same module's configuration ends up in a different language depending on whether a recipe installed it or not.

The cause looks like the syncing gate. RecipeRunner::installModules() and installTheme() wrap the install in config.installer->setSyncing(TRUE), and the resolve in ConfigInstaller::createConfiguration() sits inside

if (!$this->isSyncing() && ...)

, so it never runs for those. The recipe's own config goes through a separately constructed RecipeConfigInstaller that is not syncing, so it does resolve.

alexpott’s picture

@svendecabooter firstly thanks for taking the time out to test code on this issue. Amazing!

I would like to clarify something: It would say this issue is aiming to help resolve. I think your use-case of having the site default and config default langcodes as different is special and does not necessarily need to be fixed here.

Your feedback on Recipes and sync status is very helpful - I agree that we need to fix that. I'm not sure whether this issue should fix it (it is currently broken after all) or attempt to resolve that here to. I'm unsure if we need to resolve #3613607: Add RecipeRunner::isApplying() to determine if a recipe is being applied, without confusing it with a config sync before attempting this.

alexpott’s picture

@svendecabooter one thing the MR is introducing is an event that will allow your site to set a default config langcode to something other than the site's default langcode. Maybe, if we manage to land this issue then that's what https://www.drupal.org/project/config_language_lock can provide.

alexpott’s picture

Okay I've merged the solution from #3613607: Add RecipeRunner::isApplying() to determine if a recipe is being applied, without confusing it with a config sync into the MR and now recipes should correctly using the default langcode too. This issue is blocked on #3613607 but I want to continue working on this and hopefully that one will land quickly.

alexpott’s picture

I've added recipe testing to the new MR. @svendecabooter if you could repeat your tests that'd be amazing. Note your site will need to implement a subscriber to ConfigLangcodeResolveEvent and ensure that it has a high priorty and set the langcode to 'en'.

My next task is to update the issue summary and title to properly reflect the new MR and what is being changed by the new MR.

alexpott’s picture

Title: Introduce a dedicated "Configuration default language" different from "Site default language" » Ensure "Site default language" is also the language used when installing config via extensions and recipes and config is updated correctly when it is changed
Issue summary: View changes

I've updated the issue summary with the help of an LLM to reflect the latest MR. I'm going to hide the old MR that introduced a new setting for default config language as I no longer think that is the best way to fix the problems outlined in the issue summary.

Next I'm going to add an 11.x branch as it much easier to reproduce the problems in 11.x as locale can download translations from drupal.org there.

alexpott changed the visibility of the branch 3337864-introduce-config-default-language to hidden.

alexpott’s picture

Issue summary: View changes
alexpott’s picture

Issue summary: View changes

alexpott’s picture

Issue summary: View changes
quietone’s picture

Title: Ensure "Site default language" is also the language used when installing config via extensions and recipes and config is updated correctly when it is changed » Ensure "Site default language' is used when installing config by any means and is updated correctly

Trying for a concise title. that is still correct. For myself, less wordy title are easier to skim both in the issue queue and it git logs.

alexpott’s picture

Issue summary: View changes
svendecabooter’s picture

Thanks for merging in #3613607: Add RecipeRunner::isApplying() to determine if a recipe is being applied, without confusing it with a config sync and the further updates.
I was indeed aware of the config_language_lock module, which is how I ended up in this issue...

I have rerun my earlier test on the latest code state, and now the recipe scenario also provides the same consistent result as the others, so that appears to have fixed that issue.

So if I understand the current state of this MR correctly, it fixes (almost) all of the issues that config_language_lock describes, and "locks" the language always to the default site language (with no configuration option to choose another language).
The reason config_language_lock would then exist is to allow more advanced use cases like the one I described, where the site default language would be Dutch for example, but we'd want all config to be stored as English. Then we'd install that module and configure it to lock to English - it would do so be reacting to the new event that core submits.

One thing that remains unclear to me is how this would behave in the following scenario that has bit us a few times already in the past:
A multilingual site gets built say in English as default language. Mid-way the project the client decides they want the default language to be Dutch instead. The DefaultLangcodeConfigUpdater would then still change all config langcodes from "en" to "nl" automatically? Also when a module like config_language_lock would be configured to react to the event, as described above?
If so, that might still get us in a situation where a bunch of English strings are suddenly marked as being "Dutch", confusing website translators...

alexpott’s picture

@svendecabooter thanks for testing - it's great to know that the recipe system is now consistent - nice!.

Yes you are correctly that config_language_lock will be able to drop a lot of code once this lands and become something that solves your advanced case. And yes it will need to alter the behaviour of events that respond to system default langcode change. Actually this brings up a great point about the language config subscriber - we need to make this easier to override. I'm going to move it into it's own event so it is easier for config_language_lock to override this event and replace it with it's own logic about when to fire.

To answer your last question, if you change the default language from english to dutch this MR now ensures that all the current config values become english translations and Dutch translations are applied to the default config. The scenarios about changing the default langcode from the issue summary now behave as expected. This is quite easy to test with the 11.x branch where locale will be able to get translations from localise.drupal.org. And english strings should not be marked as Dutch.

alexpott’s picture

Issue summary: View changes

I've separated out the event that responds to a change in system.site:default_langcode so that it is easy for contrib and custom.

alexpott’s picture

Issue summary: View changes
alexpott’s picture

Issue summary: View changes

I've removed the skipping on unknown langcode from the installer - it's a different bugfix that can be handled in a follow-up. I'll create the follow-up later. I've updated the issue summary to match this change.

alexpott’s picture

Issue summary: View changes
alexpott’s picture

Issue summary: View changes
gábor hojtsy’s picture

I think this is almost there! Left a lot of review comments :) Also the issue summary needs updating for recent MR updates.

alexpott’s picture

Issue summary: View changes

I'm updating the issue summary to make it more correct. I've also removed the test coverage section as it is hard to maintain.

alexpott’s picture

Issue summary: View changes
Issue tags: -Needs issue summary update

Improve the API documentation.

alexpott’s picture

Issue summary: View changes
alexpott’s picture

Issue summary: View changes

I've tried to update the issue summary to make it clearer.

alexpott’s picture

Issue summary: View changes
alexpott’s picture

Issue summary: View changes
alexpott’s picture

Issue summary: View changes

Updated the API change section with the details of the change I've just made to support the case where there are no overrides or a partial override when you change the default langcode of a site.

phenaproxima made their first commit to this issue’s fork.

alexpott’s picture

I've closed the 11.x MR as now that 12.0.x exist we have translations for Drupal 12. Nice.

alexpott’s picture

#87 was premature you still don't get a full locale download on 12.x or main... I'll bring back the 11.x MR when I have time as it make testing this much easier.

alexpott’s picture

Created a very small self-contained issue to improve \Drupal\Core\Config\Schema\ArrayElement::hasTranslatableElements() performance - see #3627544: \Drupal\Core\Config\Schema\ArrayElement::hasTranslatableElements() creates unnecessary objects

alexpott’s picture

Issue summary: View changes

Update issue summary for the latest changes - I've managed to remove one of the events this issue was adding - LanguageDefaultLangcodeChangeEvent - locale's requirements to add to what language does all happen in the decorator.

alexpott’s picture

Issue summary: View changes
alexpott’s picture

This is ready for another round of review! Quite a few changes since the last round.

  • 1 less event - we were decorating in the locale module already so there's no need to trigger an event to get the locale module to do something else.
  • What happens when Config::save() has to add or remove a langcode has been tidied up to not rely on passing by reference - at a small and not often paid performance cost.
  • Big review of the documentation.
nicxvan’s picture

I got part of the way through reviewing this again, one question that I have after reading through this, is this changing how simple configuration works related to translation or is this just fixing the mixups that happen.

It's not clear to me if this is fixing a bug, adding new capabilities or both. I suspect it's the latter.

alexpott’s picture

@nicxvan in my mind this issue is 100% bug fixing. We already provide the capability to change your default language in core. This issue fixes bugs that occur when you do that. In order to fix the bug we've had to introduce new API around changing langcodes in config because that currently is in locale BUT you don't need locale installed to change your default language - you only need the language module. So code for managing that has moved there. Similarly, in order to set the config langcode at install time we've needed to add a new API as again it was locale that was changing configuration langcodes after module install.

alexpott’s picture

Just run an interesting test on 11.x (because you get translation files from l.d.o) ...

  • Install standard profile in english
  • Enable the language module
  • Add french
  • Set french to be the default language
  • Install the media_library module - config is create with the FR langcode - even though everything is still in english because no locale
  • Install locale
  • Check for updates (admin/reports/translations) and do the import...

French translations are applied to the config in the default config supplied y modules and en overrides are create as expected. It works great. Note this would work on HEAD but my point is that we've not broken anything. There are somethings that are different. With this MR the configuration language is changed when set french to be the default language rather by a batch triggered by the locale module install.

gábor hojtsy’s picture

Status: Needs review » Needs work

Re-reviewed my previous review notes. What I found is the early return in LocaleConfigSubscriber::onConfigSave() skips new configuration with _core.default_config_hash.

  1. Let's say the site default is Hungarian, locale is installed.
  2. On a development site, install a module. Customize a translatable value in its configuration. Export the configuration. The export contains the core hash.
  3. On production, import the configuration. The import installs the module and creates the configuration.
  4. Locale stores no customized translation for the edited value.
  5. The next locale configuration update replaces the edited value.

I worked with my LLM to propose a test for this scenario that would fail with your MR.

gábor hojtsy’s picture

Issue summary: View changes

Minor updates to issue summary. Adding a simple release notes snippet. Not sure what else to add, probably need to keep it short for anyone to read but need to be informative enough :)

alexpott’s picture

Issue summary: View changes
Status: Needs work » Needs review

Update issue summary for latest changes:

  • Removed the decorator in favour of a new DefaultLangcodeConfigReconcileEvent event triggered by \Drupal\language\DefaultLangcodeConfigUpdater::reconcileConfigLangcodeOperation and \Drupal\language\DefaultLangcodeConfigUpdater::isReconciling
  • Replaced the hook_language_preserve_config_langcode() with DefaultLangcodeConfigReconcileEvent::skip()
alexpott’s picture

I;ve also fixed the code to pass the test @gábor hojtsy added in #97

alexpott’s picture

Issue summary: View changes
alexpott’s picture

Issue summary: View changes

Fixed issues with the issue summary.

alexpott’s picture

Issue summary: View changes

Added an AI usage section to the issue summary.

needs-review-queue-bot’s picture

Status: Needs review » Needs work
StatusFileSize
new5 KB

The Needs Review Queue Bot tested this issue. It fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

gábor hojtsy’s picture

I looked at this with LLM help which I then manually reviewed :)

1. A recipe module install replaces existing locale translations with English.

This one is a new problem that was caused by the !isSyncing() check that was added for the import case I raised.

  1. The site default is Hungarian. Locale contains English test translated to Hungarian test, not customized.
  2. A recipe installs locale_test, which ships this string.

Locale string after the install:

  • MR, recipe: English test, customized=1.
  • MR, plain module install: Hungarian test, customized=0.
  • Main, recipe: Hungarian test, customized=0.

This seems to be cause by the following: RecipeRunner::installModules() setts syncing to TRUE. The early return in LocaleConfigSubscriber::onConfigSave() requires syncing to be FALSE, so it does not apply. Then ConfigInstaller sets the langcode to hu because RecipeRunner::isApplying() returns TRUE. But the value is still English. So the value is saved as the Hungarian translation because it differs from the existing translation.


2. A customized config entity (different text from the default) is not stored in locale.

Not a recently introduced problem with the MR, but is a situation where I think it behaves different from how I would expect it (IMHO unintentionally changes behaviour from main).

Steps:

  1. The site default is Hungarian. Locale contains a non-customized Hungarian translation of Default.
  2. A recipe installs a module and ships with a config entity with the label Custom (which is different from the default value Default).
  3. updateConfigTranslations() runs.

Results on the MR:

  • If the config entity file langcode is hu, then the existing translation replaces the label, even though it was not customized and should actually be overwritten. Main keeps Custom and stores it in locale as customized (which I would expect).
  • If the site default is hu and the file langcode is en or missing, then the translation replaces the label. Custom no longer exists in storage. Main keeps Custom in active config and creates a hu override for the translation. I think in this case the English was intended to be Custom should end up in a translation override, no?
  • If the site default and the file langcode are both en, then the result is the same as main.

The reason for this one is that ConfigInstaller does not create config entities while syncing is TRUE. RecipeConfigInstaller creates the entity after syncing returns to FALSE. The configuration is new and contains _core.default_config_hash. All conditions of the early return in LocaleConfigSubscriber::onConfigSave() are met, so nothing is written to locale.

I think this is hard/impossible to fix in the current design as ConfigInstaller changes en to hu before the config is saved. By the time locale handles the save, Custom with langcode: hu looks the same whether the file was really Hungarian or English. The first case should become a Hungarian customized translation, and the second an en override I think. A fix would either need to keep the file’s original langcode longer, or handle this before the relabel.


Changing the default language replace edited English value with the shipped value if there was no translation.

Steps:

  1. The site default is English, with French added.
  2. A config value is customized to My edited English value.
  3. Locale contains no French translation for this string so French uses the My edited English value too as it does not have a translation override.
  4. Site language is changed to French.

Results:

  • Active config: langcode: fr, value is the original shipped config text (wrong).
  • en override: test: My edited English value (good).

extractUntranslatedKeys() returns all translatable keys that are not in the override for the new default. This includes values in English when the old default is English. getSourceStrings() returns the shipped source for these keys. getDataForDefaultLangcodeChange() merges the source over the edited value, so values that the site did not even use anywhere unexpectedly show up.

alexpott’s picture

Status: Needs work » Needs review

I've addressed and added test coverage for all the issues raised in #105.

For point 1 - I incorporated a check of whether a recipe is being applied into the LocaleConfigSubscriber - https://git.drupalcode.org/project/drupal/-/merge_requests/16913/diffs?c...

For point 2 - I've added to the original langcode to the _core config info if it is not english - and then used this information in LocalConfigSubscriber to work out what to do - https://git.drupalcode.org/project/drupal/-/merge_requests/16913/diffs?c...

For point 3 - I no longer supplied the untranslated keys for en config - https://git.drupalcode.org/project/drupal/-/merge_requests/16913/diffs?c...

needs-review-queue-bot’s picture

Status: Needs review » Needs work
StatusFileSize
new98 bytes

The Needs Review Queue Bot tested this issue. The merge request has merge conflicts and cannot be merged. Therefore, this issue status is now "Needs work".

This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

alexpott’s picture

Status: Needs work » Needs review