Introduce a dedicated "Configuration default language" different from "Site default language"

The same issue observed in 9.5.x, it is a core design issue, I opened a multilingual support issue:
Multilingual support of strings in second language website

I Tried using configuration update manager module to adjust langcode from default language to primary language, time consuming process! Although the content is translated but it would not shown correctly in second language. Same issue here!

Updated task description to account for the fact that when you switch default language back to English, nothing happens, existing config language is never updated.

I guess we need to make locale_config_batch_set_config_langcodes() behave the same for all languages and not to make an exception if default language is "en", or maybe keep track of the last language set for configuration, and whether the default langcode has been changed or not...

Well, as far as I can see the Drupal multilingual efforts has come a long way from the time that i18n or internationalization and localization modules was once were.
To be truly multilingual the configuration translation and interface translation should be taken a second thought.

Updated task description to account for the fact that when you switch default language back to English, nothing happens, existing config language is never updated.

As you said, the the language switch should be respected both in terms of configuration of the elements of the site and also the contents. An architectural plan should be devised for this task. Currently the configuration in consistency after switching the language from default language are many to account.

I don't know if this is the right topic to share my issue, but it's also related to switching the original language.

What I did - while upgrading Drupal 8 to Drupal 9 by mistake I changed my default language from English to Spanish. When I realized this (after the upgrade) I switched my default language back to English from this page:
admin/config/regional/language

But now, when I'm trying to edit any of my translations related things it still says "Español (original)".

Is there any way to switch every translation option back to English (original)?

Made the title more specific.

Looking at why this happens. When a module is installed after the default language is changed:

• locale_modules_installed() is called (this is a hook that responds to modules being installed).
• This calls locale_system_update() which sets up a batch.
• The batch first runs translation updates (downloads .po files from localize.drupal.org and imports them). This is not relevant for this bug.
• Then it adds a sub-batch with locale_config_batch_update_components() passing the request to update default langcodes in config. Because there are no specific components requested, this will get a list of all the default config objects (config items that are shipped with installed extensions). See https://api.drupal.org/api/drupal/core%21modules%21locale%21src%21Locale...
• Then it invokes locale_config_batch_build() which uses this component list of all default config and adds two kinds of operations: (1) to update the langcode of them and (2) to update the translations of them.
• The batch operation locale_config_batch_set_config_langcodes() just wraps updateDefaultConfigLangcodes(). This operates if the site default langcode is not English and updates all of the active copies of default config that do have an English langcode or are missing a langcode to the current site default langcode. This does not change existing active config if it is already some other language. It also does not change the active config back to English if the new default is English. It just does not operate in that scenario.
• Then a batch operation is added for 20 config names each of locale_config_batch_refresh_name(). This wraps updateConfigTranslations(). This considers the active config language and detects if the translation should be stored within the active config object or an override. This is independent of the site default language and only considers the langcode of the active config object.

All in all I think the logic protects the existing language changes in active config if they were made prior to the default language change. So if you already have a Swedish site let's say, you switch to Finnish, the active config items that were previously Swedish will remain Swedish. Newly installed modules will get their config in Finnish in active config, so you end up with a mix of config items with Swedish and Finnish. But the integrity of your config is maintained.

If you built out a part of the website in English and then switch the default langcode, this indeed ends up doing more dramatic changes later because English is considered special. (When the default language is set back to English, I think the data protection logic does what it was supposed to do).

Now as to what happens when default language is changed, I'll post another comment, this is already quite long :D

Default language change on the UI is implemented in LanguageListBuilder::submitForm(), and does this essentially:

   // Save the default language if changed.
    $new_id = $form_state->getValue('site_default_language');
    if ($new_id != $this->languageManager->getDefaultLanguage()->getId()) {
      $this->configFactory->getEditable('system.site')->set('default_langcode', $new_id)->save();
      $this->languageManager->reset();
    }

    if ($this->languageManager instanceof ConfigurableLanguageManagerInterface) {
      $this->languageManager->updateLockedLanguageWeights();
    }

Language module reacts on the config save of the default langcode in its ConfigSubscriber but that of course is not related to interface translation.

Finally locale has its own LocaleConfigSubscriber but that does not consider a default language code change as something to care about.

I'm a bit torn as to how to resolve this. For one, we want to make default language changing as easy as possible. The intent of the code is to consider config similar to content, where the existing language and overrides of configs is kept, except in the English special case that it assumes belongs to the default config that needs changing. There is no such logic for content entities either to swap their default language and translations when a site default language changes.

At the same time, users likely expect to edit configuration in the site's default language by default, which the current logic attempts to ensure, at least as long as that default is English and after the first transition from English to non-English. It does not maintain that user experience once you want to make the second language transition, whether back to English or to something else. It does leave your config intact but it does not maintain the assumed user experience. Also, if the first language transition happens later in the site's life, a massive config change happens then. That in itself is expected, but not when you install a new module :D So the problems are:

• When you change the site default language, the current logic is not run. That would probably be "easy" to fix by launching the existing batch then, since it deals with all default config. Not sure how to make it happen in a config import though?
• After you change the site default language AGAIN, the current logic keeps your config as-is (not English) and new config is now created in the new default language. To maintain the idea that your active config is in the site default by default, we would need the logic explained in the issue summary to extract the translations from active config, create the overrides and save the active config with the proper values fom the overrides in the new default (and remove the new default's overrides for default config).

It seems that someone has been changing some things in how languages work.

I am now unable to rename a certain language (D 10.2.4).
I wanted to name English en and German de.
The YML configuration is correct, but when the page is shown in the default original language German, it will just say "Deutsch" no matter what.
I guess the new code changes fail to retrieve and display the correct label.

Actually I had it working correctly on a site where I did the following :

Installed the site in English

Added German
Added British English
Deleted English

works.

there is actually more to this issue.

languages need a translatable label, which would resolve the present issue and a dozen more related issues I have found today when going down this rabbit hole.

BUT
languages also need editable labels that will be displayed in translations, verbose translations and abbreviated translations, URLs ...

So for example, a site could have installed British English and it would display the following way :
"en-gb" (non translatable) in any technical-related stuff such as hreflang, html, etc.
"British English" (translatable) in any configuration dialog in the admin interface, translation forms, etc.
"en" (translatable) as a label for language switcher links
and potentially, the site owner would want something else (translatable) to be displayed in the URL, such as /eng/ or /english/ or ... whatever

so the short ISO language code would be used by Drupal to identify the language and source translations, and would be untranslatable.
and then there should be 3 translatable labels on top.

I can now also confirm the workaround to correct this issue on running sites.

First, install an additional language.
Then change the language of all localized content to that new language you just installed.
That may be nodes, blocks, media, redirects and URLs, etc.

Then delete the language of which you can't change the name (be careful, this will delete all content you forgot to switch to the new language).

Purge the cache and then re-install the language you just deleted.

Edit the name of that language to what you wanted and then change the contents' language back to what it was.

So it seems like something screws up the language settings when installing the site or when declaring a language default.

never mind the workaround, one of the language labels reverted back to original after doing something else on the site.

I don't know whether is related to this. I set the default language to other than English. Then I've exported the config (drush cex). Reinstalled the site from config but the English language is gone. Checked the config sync directory, I can see the language.entity.en.yml file. Checked the {config} table, no language.entity.en at all. It just vanished.

EDIT: Ignore this comment. I just discovered that I need to set keep_english: true in the installed profile. Maybe this deserves to be better documented, even by showing a warning message to the site builder when they are changing the default language?

EDIT2: I have created #3545306: Installation from config removes English if the installed profile does not have keep_english for this issue

Nice, just found this and I think it's very similar to this one: #3568743: Convert / unify all default configs to the site default language (which has a possible solution).

For the future I'd vote to always store the default config in English by design, so we can't have such cases any more and treat any other language as translations in their respective collection / folder!

That will also simplify importing (typically English) config from modules etc...

Added to the "Refine settings, flexibility and coherences between translatable areas" outline on git.drupalcode.org. Thoughts appreciated.

Connecting related issue and adding "Update Install a language doc page" in Issue Summary as a task.

@anybody: in some European studies (eg a study of 100k websites of in the EU, 4-5k sites per country) it was found that 80% of the sites are not English single language. Why would they need to create content types, views, etc. in English? That would be bad :) Drupal already takes some effort to keep the default config in your language, we need to fix the missing use cases.

I wrote out some plans above 2 years ago in #9, #10 and #11. Since then I realized we probably don't need to deal with this in config import as that would assume you are already deploying an updated set of config. So we need to do this on the form. Since that already happens later when you install a module, we should at first bring the same behaviour forward to the default language change IMHO as a first step.

@gábor hojtsy thanks. Yes I understand that. My recurring issue in the role of a site builder is, that again and again I have to think about in which language I'm acting and how I should name even the machine names.

In 60% of our cases the projects are in German, some of them are ONLY in German and some of them are translatable into other languages. But we love the configs to be consistently in English as base language..

From our perspective it would be more appropriate to be able to rely on English as base language and use translation for anything else.
You know a lot more about the internal concepts and I think you discussed this 100000x more than me and have good reasons. So this comment is not meant to tell what should be done, just to let you know about the background and our daily pain with the "mix" we're experiencing.

Still my solution would not save us from having clients that then create entity types with German names ;) So there's no perfect. Most importantly the bug needs to be fixed, as you wrote. For the other things we'll still have to find a better / best practice.

Thank you so much! :)

I'm working on implementing this now :)

@andybody: for your base English use case you can install the site in English and keep site default in English, why not do that?

While I pushed a complete lifecycle solution above in an MR, I decided that this will not be the best route :D Instead I think there is a lot of merit in trying to cater more for what @anybody is saying about consistent source language AND decouple the config default language from site default after that (so changing config default language is a conscious choice separate from site default). For the first part of this, I opened #3608533: Introduce "Configuration language" settings page to control langcode visibility. Once that lands, I think we can return here and introduce the Config default language and adjust the code to use that when dealing with config.

I'll work on #3608533: Introduce "Configuration language" settings page to control langcode visibility tomorrow, marking this postponed on that. I already posted the rationale and high level plan there though :)

@gábor hojtsy re #25 we do that in many cases, but it doesn't feel perfect because then you have the English language also in many Content (related) areas. It just doesn't feel natural and correct to me. And you still have to be very careful to not accidentally use the wrong language somewhere.

#27 sounds like you came to a similar conclusion. Thank you so much for all your work put into this.

@anybody: yeah I did came to the conclusion that we need to decouple the "Configuration default language" from the "Site default language", which is another way to approach "enforce a config language". I also came to the conclusion that the few core changeable config languages are in the way sort of :D So I created #3608533: Introduce "Configuration language" settings page to control langcode visibility and proposed a solution there. That introduces explicit intent about language of those config, so after that as we want to I think introduce a dedicated "Configuration default language" here, we can rely on that intent / configuration to not change things that site owner explicitly wanted to be in other languages but change the rest of the config as needed. (Also to not change config language on site default language change ever). Let's reposition this issue for that, but we need to resolve #3608533: Introduce "Configuration language" settings page to control langcode visibility so we have data on the intent of the config that is currently potentially in other languages. That also helps enforce a consistent language if you don't configure those entities to be able to pick a language which is the new core default as proposed there.

Please review that one to push this cause forward :)

Created a parent issue to group related problems and show how this leads to resolving config language issues once and for all hopefully :D

Rewrote the issue summary. Will of course need more expansion but this is a good start IMHO.

Fix table HTML.

Fix one more HTML issue and reorder the events in the HTML to make it easier to follow.

More elaborate items in the table to point out the problems better.

Fix list markup.

Massive update to issue summary based on recent discoveries. I found 3 new problems since the above, so getting more convinced that we want to have one unified "Configuration default language".

Clarified a few items.

Thank you for this :-)

I was tracing a situation that I wasn't sure is 'my' bug or not, and I haven't seen it mentioned anywhere:

The samlauth module starts out with very minimal installed config, with no label values and no language code. But there are labels in the config object, and as soon as you set a label value... Configuration Inspector reports a missing language code. #3611776: samlauth.authentication config object must specify a language code

It felt like either I was missing how exactly I should solve this... or the only way to solve that reliably would be adding code into Config::save(). (Which felt like too big a thing to propose for someone without intricate knowledge of the config system...)

Luckily, I read your latest MR's code and it does just that! So... I believe I bumped into yet another issue that is solved by your MR!

@roderik: I have a targeted issue for that problem at #3600904: Shipped configuration entities that do not have translatable elements are not set to the site default langcode as that is also reproducible with core only :)

I've been experimenting how far this needs to be stretched in the past few days and ended up building https://www.drupal.org/project/config_language_lock which has about 900 lines of PHP code and almost 2x the amount of that in docs and 4x the amount that in tests to prove how core works vs how the module works. People following this issue would be welcome to test that (in dev environments for now, it is very early stages). That said the docs and tests and SUPER thorough, so I'm fairly confident it is fine. Read the very extensive docs at https://project.pages.drupalcode.org/config_language_lock/

D'oh! I failed to read this issue's meta parent and its children.

Reviewed #3600904: Shipped configuration entities that do not have translatable elements are not set to the site default langcode. The issue I stumbled into is not currently solved by it though (because it's simple config), so curious for the next step.

It would be useful for installer refactoring if we were able to move the default language configuration out of system module to core/

This is because if you try to install system module along with any other modules at the same time, it's impossible to set the system default language in between system module being installed and the other modules getting installed, and that information comes from the request.

Currently working around this in #3614153: Reduce container rebuilds in the installer where you can see having the config outside system would make it a lot easier.