Problem/Motivation

Creating languages right now is very inconsistent. I started looking into it because we have an issue in our install profile that the language was English and wouldn't update automatically when importing translations. What followed is a fun ride through language module and its history ;)

1. When installing Drupal in a specific language, the language label is created in English while everything else (as much as possible) is translated. This is related to #2031197: Language configuration entities should be created in English at all times, but that is a very old issue and we do *not* create them with langcode en anymore, the argument there is also wrong about selecting from an english list, see 2.. What we should do at that point is create the entity with the localized language label, which we do have access to.

2. When then adding a second language, the language label list is being translated to the current language, so I see e.g. "Französisch" there.

3. When then adding that language, it is again created with the english label and the config entity has the langcode of my current language (de in my case). At that point, I would expect that it is created as "Französisch" since langcode is de. It could also be in french, if the langcode would be french. Optimaly, we'd also already create a french translation after importing the french translations.

4. When I now edit that language and save it, whether or not I change the label, the langcode is changed to fr. That's because of some strange/old code in \Drupal\language\Form\LanguageFormBase that defines a hidden langcode property, which then accidently overwrites the langcode when it is actually supposed to be a hidden property for the ID. This predates the rename of the langcode property to id on the language config entity. There is also no reason to have that in the first place, this is an entity form and we have the entity object already available.

Proposed resolution

1. Create the language entity in the installer with the localized language code.
3. Create additional languages with the translated label into the correct language (==langcode property). Look into creating a translation. Maybe that could be a follow-up because it is likely more complicated.
4. Remove the langcode property for existing language entities to avoid accidently overriding that.

Remaining tasks

User interface changes

API changes

Data model changes

CommentFileSizeAuthor
#36 2927338-36.patch7.84 KBBerdir
#32 2927338-32.patch7.82 KBswatichouhan012
#30 language-labels-2927338-30.patch7.88 KBBerdir
#27 language-labels-2927338-27.patch7.79 KBBerdir
#25 interdiff_23_25.txt736 bytesanmolgoyal74
#25 language-labels-2927338-25.patch7.77 KBanmolgoyal74
#23 language-labels-2927338-23.patch7.78 KBBerdir
#19 language-labels-2927338-19.patch7.78 KBBerdir
#16 language-labels-2927338-16.patch7.8 KBBerdir
#14 language-labels-2927338-14-interdiff.txt4.07 KBBerdir
#14 language-labels-2927338-14.patch7.79 KBBerdir
#10 language-labels-2927338-10-interdiff.txt1.71 KBBerdir
#10 language-labels-2927338-10.patch7.64 KBBerdir
#4 language-labels-2927338-4-interdiff.txt1.93 KBBerdir
#4 language-labels-2927338-4.patch8.34 KBBerdir
#2 language-labels-2927338-2.patch6.41 KBBerdir
#2 language-labels-2927338-2-test-only.patch2.55 KBBerdir
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Berdir created an issue. See original summary.

Berdir’s picture

Updated an existing test with a failing assertion for each problem and fixed it, except the part about adding a translation automatically as well.

Status: Needs review » Needs work

The last submitted patch, 2: language-labels-2927338-2.patch, failed testing. View results

Berdir’s picture

Status: Needs work » Needs review
FileSize
8.34 KB
1.93 KB

Fixed the other installer test fails. 3/4 that assert for German failed where it is now Deutsch, for one it is still German because that is importing the language from default config.

Gábor Hojtsy’s picture

Issue tags: +D8MI, +language-config

We discussed this quite a bit yesterday. I have some vague recollection that the config entities may be mostly English due to some race-condition or inter-dependency with config language handling maybe. Like if the language of your config is XY and you just define XY with that config itself, it may cause problems. But not sure if that is the case or if we documented that somewhere. There is clearly no tests to ensure that or if this would fail or need to modify that test :D

Berdir’s picture

Assigned: Unassigned » alexpott

Yeah, as discussed in IRC, possibly those problems existed at some point but I think they no longer do.

Gabor suggested to ask alex, so assigning to him, lets see if that works :)

alexpott’s picture

Assigned: alexpott » Unassigned
Status: Needs review » Needs work
  1. Not sure if I remember any reason why this is as complex as it is. I guess we need better test coverage of the expectations.
  2. +++ b/core/includes/install.core.inc
    @@ -1602,8 +1602,27 @@ function install_download_additional_translations_operations(&$install_state) {
    -    // Create the language if not already shipped with a profile.
    -    $language = ConfigurableLanguage::createFromLangcode($langcode);
    

    Rather than not using createFromLangcode() how about just using setName() on the result? Less logic here and perhaps easier to comment on.

  3. +++ b/core/includes/install.core.inc
    @@ -1602,8 +1602,27 @@ function install_download_additional_translations_operations(&$install_state) {
    +      // Drupal does not know about this language, so we set its values with the
    +      // best guess. The user will be able to edit afterwards.
    +      $language = ConfigurableLanguage::create([
    +        'id' => $langcode,
    +        'label' => $langcode,
    +      ]);
    

    Needs testing I guess. Never seen a test for installing in a non-existent langcode.

Berdir’s picture

Thanks for the review.

1. Ok. This does add explicit test coverage for the 3 things that are changing, is that enough?
2. I can try that. I still need to get the list of languages and check if it exists, but..
3. Thinking about that, I don't think that is even possible, at least not in the UI. I just copied the code without thinking too much. But with the change from above, this will go away anyway, we'd simply optionally set the native label if we have one for the given language code.

Gábor Hojtsy’s picture

The installer takes the predefined language list from core + if there is any .po file locally which is added onto the list AFAIR.

Berdir’s picture

Status: Needs work » Needs review
FileSize
7.64 KB
1.71 KB

This is now again calling the API and just sets the label if we have one, as suggested.

Status: Needs review » Needs work

The last submitted patch, 10: language-labels-2927338-10.patch, failed testing. View results

Berdir’s picture

Status: Needs work » Needs review

Was just a testbot issue, this passed.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Berdir’s picture

Reroll, conflicts on tests, also updated the lines I touched to use the new assertion methods.

Status: Needs review » Needs work

The last submitted patch, 14: language-labels-2927338-14.patch, failed testing. View results

Berdir’s picture

Status: Needs work » Needs review
FileSize
7.8 KB

Patch above is for 8.5.x, this should apply to 8.6.x. trailing commas were added on the code that I'm removing anyway here.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Berdir’s picture

Reroll for 8.7.x.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Berdir’s picture

Status: Needs review » Needs work

The last submitted patch, 23: language-labels-2927338-23.patch, failed testing. View results

anmolgoyal74’s picture

Status: Needs work » Needs review
FileSize
7.77 KB
736 bytes

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Berdir’s picture

And another reroll for 9.2, those assert patches are going to cause a lot of conflicts...

alexpott’s picture

This looks pretty sensible. One thing I wonder is what should we do if the installer option keep_english is set to TRUE. Atm we leave the english language created during the install of the language module alone. I wonder if we should really be changing to be an english in the language of the site install.

And there are the other "languages" provided by the language module - UND and ZXX what about them?

alexpott’s picture

Oh that's funky if you set keep_english to TRUE then the resulting config in the default collection is correct...

uuid: 0e907c54-2789-47dd-a188-826bb494f40c
langcode: de
status: true
dependencies: {  }
_core:
  default_config_hash: lBXDpdDPXQtrfTJQhr6MjRJJEEyYSoRJ0acdvHLsWeA
id: en
label: Englisch
direction: ltr
weight: 0
locked: false

and the language/en has the expected override...

label: English

Same is true for und and zxx languages so the approach in this issue is definitely correct.

Berdir’s picture

Reroll for latest 9.2 and I guess also 9.3.x.

Re #29, does that mean we're good then here? :)

rkoller’s picture

Status: Needs review » Needs work

i've tried to apply the patch from comment #30 to Drupal 9.3.0-beta2 but composer patches was unable to apply.

swatichouhan012’s picture

Regenerated #30 patch.

swatichouhan012’s picture

Status: Needs work » Needs review

Regenerated #30 patch.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Berdir’s picture

Another reroll.

smustgrave’s picture

Status: Needs review » Needs work
Issue tags: +Needs Review Queue Initiative

This issue is being reviewed by the kind folks in Slack, #needs-review-queue-initiative. We are working to keep the size of Needs Review queue [2700+ issues] to around 400 (1 month or less), following Review a patch or merge request as a guide.

Does this require a change record?

Apologize if I missed it but has

Create additional languages with the translated label into the correct language (==langcode property). Look into creating a translation. Maybe that could be a follow-up because it is likely more complicated.

been addressed or a follow up made?

Also are the steps in the IS still the best way to test this?

Berdir’s picture

Status: Needs work » Needs review

There's no follow-up, but I'm not sure about if we even need to do that. The first part happens here, it's only that we could also create a translation already of that language, but that sounds more a feature than a bugfix to me.

The steps should still be accurate and this still applies.

smustgrave’s picture

Status: Needs review » Reviewed & tested by the community

Fun seeing my boilerplate message from when the #needs-review-queue started

Thanks @berdir for following up.

Verified the tests fail without the fixes
Applied the patch #36
Did an install of Standard profile with a different language other then English and no issues there
Added another language without issue.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 36: 2927338-36.patch, failed testing. View results

Version: 10.0.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.