Problem/Motivation
Creating languages right now is very inconsistent. I started looking into it because we have an issue in our install profile that the language was English and wouldn't update automatically when importing translations. What followed is a fun ride through language module and its history ;)
1. When installing Drupal in a specific language, the language label is created in English while everything else (as much as possible) is translated. This is related to #2031197: Language configuration entities should be created in English at all times, but that is a very old issue and we do *not* create them with langcode en anymore, the argument there is also wrong about selecting from an english list, see 2.. What we should do at that point is create the entity with the localized language label, which we do have access to.
2. When then adding a second language, the language label list is being translated to the current language, so I see e.g. "Französisch" there.
3. When then adding that language, it is again created with the english label and the config entity has the langcode of my current language (de in my case). At that point, I would expect that it is created as "Französisch" since langcode is de. It could also be in french, if the langcode would be french. Optimaly, we'd also already create a french translation after importing the french translations.
4. When I now edit that language and save it, whether or not I change the label, the langcode is changed to fr. That's because of some strange/old code in \Drupal\language\Form\LanguageFormBase that defines a hidden langcode property, which then accidently overwrites the langcode when it is actually supposed to be a hidden property for the ID. This predates the rename of the langcode property to id on the language config entity. There is also no reason to have that in the first place, this is an entity form and we have the entity object already available.
Proposed resolution
1. Create the language entity in the installer with the localized language code.
3. Create additional languages with the translated label into the correct language (==langcode property). Look into creating a translation. Maybe that could be a follow-up because it is likely more complicated.
4. Remove the langcode property for existing language entities to avoid accidently overriding that.
Remaining tasks
User interface changes
API changes
Data model changes
Comment | File | Size | Author |
---|---|---|---|
#36 | 2927338-36.patch | 7.84 KB | Berdir |
| |||
#32 | 2927338-32.patch | 7.82 KB | swatichouhan012 |
#30 | language-labels-2927338-30.patch | 7.88 KB | Berdir |
#27 | language-labels-2927338-27.patch | 7.79 KB | Berdir |
#25 | interdiff_23_25.txt | 736 bytes | anmolgoyal74 |
Comments
Comment #2
BerdirUpdated an existing test with a failing assertion for each problem and fixed it, except the part about adding a translation automatically as well.
Comment #4
BerdirFixed the other installer test fails. 3/4 that assert for German failed where it is now Deutsch, for one it is still German because that is importing the language from default config.
Comment #5
Gábor HojtsyWe discussed this quite a bit yesterday. I have some vague recollection that the config entities may be mostly English due to some race-condition or inter-dependency with config language handling maybe. Like if the language of your config is XY and you just define XY with that config itself, it may cause problems. But not sure if that is the case or if we documented that somewhere. There is clearly no tests to ensure that or if this would fail or need to modify that test :D
Comment #6
BerdirYeah, as discussed in IRC, possibly those problems existed at some point but I think they no longer do.
Gabor suggested to ask alex, so assigning to him, lets see if that works :)
Comment #7
alexpottRather than not using createFromLangcode() how about just using setName() on the result? Less logic here and perhaps easier to comment on.
Needs testing I guess. Never seen a test for installing in a non-existent langcode.
Comment #8
BerdirThanks for the review.
1. Ok. This does add explicit test coverage for the 3 things that are changing, is that enough?
2. I can try that. I still need to get the list of languages and check if it exists, but..
3. Thinking about that, I don't think that is even possible, at least not in the UI. I just copied the code without thinking too much. But with the change from above, this will go away anyway, we'd simply optionally set the native label if we have one for the given language code.
Comment #9
Gábor HojtsyThe installer takes the predefined language list from core + if there is any .po file locally which is added onto the list AFAIR.
Comment #10
BerdirThis is now again calling the API and just sets the label if we have one, as suggested.
Comment #12
BerdirWas just a testbot issue, this passed.
Comment #14
BerdirReroll, conflicts on tests, also updated the lines I touched to use the new assertion methods.
Comment #16
BerdirPatch above is for 8.5.x, this should apply to 8.6.x. trailing commas were added on the code that I'm removing anyway here.
Comment #19
BerdirReroll for 8.7.x.
Comment #23
BerdirReroll.
Comment #25
anmolgoyal74 CreditAttribution: anmolgoyal74 at OpenSense Labs for DrupalFit commentedComment #27
BerdirAnd another reroll for 9.2, those assert patches are going to cause a lot of conflicts...
Comment #28
alexpottThis looks pretty sensible. One thing I wonder is what should we do if the installer option
keep_english
is set to TRUE. Atm we leave the english language created during the install of the language module alone. I wonder if we should really be changing to be an english in the language of the site install.And there are the other "languages" provided by the language module - UND and ZXX what about them?
Comment #29
alexpottOh that's funky if you set keep_english to TRUE then the resulting config in the default collection is correct...
and the language/en has the expected override...
Same is true for und and zxx languages so the approach in this issue is definitely correct.
Comment #30
BerdirReroll for latest 9.2 and I guess also 9.3.x.
Re #29, does that mean we're good then here? :)
Comment #31
rkolleri've tried to apply the patch from comment #30 to Drupal 9.3.0-beta2 but composer patches was unable to apply.
Comment #32
swatichouhan012 CreditAttribution: swatichouhan012 at QED42 for QED42 commentedRegenerated #30 patch.
Comment #33
swatichouhan012 CreditAttribution: swatichouhan012 at QED42 for QED42 commentedRegenerated #30 patch.
Comment #36
BerdirAnother reroll.
Comment #37
smustgrave CreditAttribution: smustgrave at Mobomo commentedThis issue is being reviewed by the kind folks in Slack, #needs-review-queue-initiative. We are working to keep the size of Needs Review queue [2700+ issues] to around 400 (1 month or less), following Review a patch or merge request as a guide.
Does this require a change record?
Apologize if I missed it but has
Create additional languages with the translated label into the correct language (==langcode property). Look into creating a translation. Maybe that could be a follow-up because it is likely more complicated.
been addressed or a follow up made?
Also are the steps in the IS still the best way to test this?
Comment #38
BerdirThere's no follow-up, but I'm not sure about if we even need to do that. The first part happens here, it's only that we could also create a translation already of that language, but that sounds more a feature than a bugfix to me.
The steps should still be accurate and this still applies.
Comment #39
smustgrave CreditAttribution: smustgrave at Mobomo commentedFun seeing my boilerplate message from when the #needs-review-queue started
Thanks @berdir for following up.
Verified the tests fail without the fixes
Applied the patch #36
Did an install of Standard profile with a different language other then English and no issues there
Added another language without issue.