Problem/Motivation

That method attempts to keep all config translation as well as the active configuration in sync with interface translation.

The problem is when you have EN enabled but not as the default language. That logic treats it as any other language, but it's different in a number of ways, for example that even though you can enable local EN translation, localize.drupal.org will never provide any translations for it.

Also, locale/interface translation is always EN => EN, and it implicitly assumes that is the case case for config as well, so it thinks that if there is no EN-specific translation for e.g. "Page", then it can clean up the config override as well and it will fall back to the hardcoded version. But there is no hardcoded version and your active config is e.g. DE and you need to have that EN version of it.

Proposed resolution

Completely skip locale => config sync for EN. That will change the behavior of EN overrides compared to other languages a bit. There you can e.g. translate prev/next labels to your language in one view, and then that will sync that into interface translation and then you can force it to sync that back into all your other views. That's fairly unpredictable though as it only happens in specific scenarios (enabling a module, importing translations or doing a full translation sync from localize.drupal.org)

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

Comments

Berdir created an issue. See original summary.

berdir’s picture

Status: Active » Needs review
StatusFileSize
new1.13 KB

First patch, wondering if anything explicitly tests for this.

Status: Needs review » Needs work

The last submitted patch, 2: 3079242-2.patch, failed testing. View results

berdir’s picture

Status: Needs work » Needs review
StatusFileSize
new1.89 KB
new2.85 KB

So, this does have specific tests that explain the use case, that's good :)

so we do need this sync to for example add default shipped labels in to EN overrides when EN is enabled and later on new config is imported. That makes sense.

I think the fix can be more specific then, I was debugging it and basically, it's the filtering:

$data = $this->filterOverride($override->get(), $translatable);

That removes everything from the data that exists in the shipped config as a key, so that it will then get re-added from $processed. But as I mentioned before, this assumption doesn't really work for EN, so I added a check for EN there.

With this understanding, I was also able to reproduce this quite easily in \Drupal\Tests\locale\Kernel\LocaleConfigSubscriberForeignTest::testEnglish(), I just need to call updateConfigTranslations() again and the EN override is removed.

I did have to include one part of #3079330: LocaleConfigSubscriberTest has many assertions that don't run as it otherwise never runs the actually relevant assertion and doesn't fail.

berdir’s picture

Title: \Drupal\locale\LocaleConfigManager::updateConfigTranslations() should not try to update EN » \Drupal\locale\LocaleConfigManager::updateConfigTranslations() should not remove non-existing EN translations

The last submitted patch, 4: 3079242-language-config-sync-en-4-tests-only.patch, failed testing. View results

gábor hojtsy’s picture

I can see how for config that was shipped in English but then turned to be a foreign language due to how the site was set up, yet you still want an English variant, you don't want that English variant to disappear just because it may be the same as the original shipped config. So I think I understand the problem.

But! While localize.drupal.org does not provide English translations, it is entirely feasible in a Drupal deployment scenario that people would deploy English translations for stuff with .po files, that is at least an existing core feature. This change would not be backwards compatible with the capabilities offered currently. Is there a way to target the bug fix better at when the active config is not English or would not fall back on the string we expect it would fall back on, we don't remove that from the config.

berdir’s picture

I have no idea how we'd detect that?

The remove/filter feature is specifically about removing strings in config that were removed from the locale tables, but it's impossible for us to detect if something never existed or was removed? We'd need to keep removed translations in the database and flag them as deleted

To be clear, the only "feature" we break is the cleanup of removed translations, adding new ones should still work fine.

gábor hojtsy’s picture

I am a bit uneasy about this patch because it sounds like we keep the values in place for this process. But then when you save new locale strings for example, is it also kept then. The config and locale translations are supposed to be continually kept in sync and this changes that in one place. So the process can still come around and hurt you elsewhere. I don't know exactly where yet, but I think we need to consider more angles.

berdir’s picture

> But then when you save new locale strings for example, is it also kept then.

Not sure if I understand you correctly. If you *do* have an EN translation, then it's still being updated, also if you change them, on the line below:

$data = NestedArray::mergeDeepArray([$data, $processed], TRUE);

So IMHO the only downside here is that we no longer do cleanup of old keys if they really should be removed. But that seems like a tiny problem compared to having dozens of english overrides deleted whenever I do a module installation.

I created this as major, but I think if the other two issues that we've been talking about are critical then this is just as bad and for affected sites not just a one-time thing when switching the language but happens every time this sync process runs.

That said, I'm more than happy to give this some time and wait for other reports. I've tested it on one site now, where it works well and included it in our distribution, but we only have two sites currently with this setup (english enabled and not the default language).

gábor hojtsy’s picture

Hm, where do we ensure that we don't save back translations to overrides for keys that don't exist anymore in the active config? Let's say you remove a display from a view, the config gets saved so locale jumps in to update its translations. If the English translations would still be kept in the override for the config, even though the view display is not anymore in place, that is a problem. When the English translation is requested the phantom items will get array merged on top of it and you would get a broken view config loaded with stray keys from a display that does not exist anymore.

alexpott’s picture

+++ b/core/modules/locale/src/LocaleConfigManager.php
@@ -584,7 +584,7 @@ public function updateConfigTranslations(array $names, array $langcodes = []) {
           // Filter out locale managed configuration keys so that translations
           // removed from Locale will be reflected in the config override.
-          $data = $this->filterOverride($override->get(), $translatable);
+          $data = $langcode != 'en' ? $this->filterOverride($override->get(), $translatable) : $override->get();

Some more commentary would be helpful here.

I think @Gábor Hojtsy's point in #11 is valid - what happens if the old keys have an effect like in the example given. If there no way to detect if old keys

really should be removed

OTOH I also agree that the data loss on module install is as serious so I'm +1 on calling this critical.

ayalon’s picture

Thanks @berdir for a fix for this long standing issue.

I basically only work with non-english Drupal installation. When ever I installed a module, tons of translations in views and other config files got reverted to english.
This patch finally fixes this issue. I'm happy that from now on I only have to care about "changed" config files and not manually have to revert overridden config files.

Thanks a lot!

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

The last submitted patch, 15: 3079242-language-config-sync-en-15-TEST-ONLY.patch, failed testing. View results

Status: Needs review » Needs work

The last submitted patch, 15: 3079242-language-config-sync-en-15.patch, failed testing. View results

primsi’s picture

Seems like the above patch failed, because after #3079330: LocaleConfigSubscriberTest has many assertions that don't run in \Drupal\Tests\locale\Kernel\LocaleConfigSubscriberTest::assertNoConfigOverride the second assert actually runs.

This patch then fails in $this->deleteLocaleTranslationData($config_name, 'test', 'English test', 'en'); which also calls $this->assertNoConfigOverride() (side note: I guess calling testEnglish() after deleteLocaleTranslationData() in \Drupal\Tests\locale\Kernel\LocaleConfigSubscriberForeignTest::testEnglish is redundant?).

To add to the confusion the arguments passed to \Drupal\Tests\locale\Kernel\LocaleConfigSubscriberTest::assertNoConfigOverride are wrong in all the places where called except in the test we are changing (\Drupal\Tests\locale\Kernel\LocaleConfigSubscriberForeignTest::testEnglish).

andypost’s picture

+++ b/core/modules/locale/src/LocaleConfigManager.php
@@ -584,7 +584,7 @@ public function updateConfigTranslations(array $names, array $langcodes = []) {
           // Filter out locale managed configuration keys so that translations
           // removed from Locale will be reflected in the config override.
-          $data = $this->filterOverride($override->get(), $translatable);
+          $data = $langcode != 'en' ? $this->filterOverride($override->get(), $translatable) : $override->get();

Also comment should be extended about why 'en' used... some sites can have no such langcode

primsi’s picture

Created an issue for the wrong arguments mentioned in #18: #3126195: Fix LocaleConfigSubscriberTest::assertNoConfigOverride method

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

primsi’s picture

Status: Needs work » Needs review
StatusFileSize
new2.38 KB
new8.18 KB

I am trying with the patch from #3126195: Fix LocaleConfigSubscriberTest::assertNoConfigOverride method. Test were still failing locally though. So I replaced the deleteLocaleTranslationData() call with what that method does minus the ending assert. Because if I correctly understand this patch, wouldn't we then expect the en translation not to be deleted? I might be wrong though, because I am not that familiar with the issue.

I also added the comment mentioned in #19. I used #7 as reference.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

needs-review-queue-bot’s picture

Status: Needs review » Needs work
StatusFileSize
new185 bytes

The Needs Review Queue Bot tested this issue. It either no longer applies to Drupal core, or fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

Apart from a re-roll or rebase, this issue may need more work to address feedback in the issue or MR comments. To progress an issue, incorporate this feedback as part of the process of updating the issue. This helps other contributors to know what is outstanding.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.