Problem/Motivation

When loading the front page of Umami with an empty cache, there are dozens of queries for configuration language overrides, despite the 'translate_english' setting in locale being off by default.

This accounts for something like 60 unnecessary queries, so marking major.

Steps to reproduce

Proposed resolution

Add a language.translate_english container parameter - this will default to true to preserve existing behaviour. It could potentially be changed in a follow-up to default to 'false'.

locale will then flip it to false when locale's translate_english setting is set to 'false'.

Remaining tasks

User interface changes

Introduced terminology

API changes

Data model changes

Release notes snippet

Issue fork drupal-3518992

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

catch created an issue. See original summary.

catch’s picture

Status: Active » Needs work

Pushed a proof of concept, including #3408713: Add database and cache assertions to OpenTelemetryFrontPagePerformanceTest and OpenTelemetryNodePagePerformanceTest which makes it easy to see the difference.

To do this properly we'd want to inject the config factory and not load the locale.settings config in the constructor in case it's not used - can have a method to get it instead.

catch’s picture

    
    Open Telemetry Front Page Performance (Drupal\Tests\demo_umami\FunctionalJavascript\OpenTelemetryFrontPagePerformance)
     ✘ Front page performance
       ┐
       ├ Failed asserting that two arrays are identical.
       ┊ ---·Expected
       ┊ +++·Actual
       ┊ @@ @@
       ┊  Array &0 [
       ┊ -····'QueryCount'·=>·376,
       ┊ -····'CacheGetCount'·=>·471,
       ┊ -····'CacheSetCount'·=>·467,
       ┊ +····'QueryCount'·=>·298,
       ┊ +····'CacheGetCount'·=>·393,
       ┊ +····'CacheSetCount'·=>·389,
       ┊      'CacheDeleteCount' => 0,
       ┊      'CacheTagLookupQueryCount' => 49,
       ┊      'CacheTagInvalidationCount' => 0,

Works.

catch’s picture

Status: Needs work » Needs review

Tidied up the MR. Still includes #3408713: Add database and cache assertions to OpenTelemetryFrontPagePerformanceTest and OpenTelemetryNodePagePerformanceTest but that issue is RTBC, so moving to needs review.

catch’s picture

https://git.drupalcode.org/project/drupal/-/merge_requests/11822/diffs?c... shows the performance improvement. About 90 database queries and cache gets removed on a cold cache.

berdir’s picture

That setting is only for interface translation, if your config language is not en you need to load english overrides. But maybe we can check the lang code instead?

catch’s picture

hmm good point, so something like check if the config langcode is the same as the default langcode and short-circuit that way?

andypost’s picture

There's many bugs in config translation and locale so I'm not sure we can guess anything until the flow is settled

IMO the cause here is #3150540: Configuration langcode is forced to site default language

needs-review-queue-bot’s picture

Status: Needs review » Needs work
StatusFileSize
new91 bytes

The Needs Review Queue Bot tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".

This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

catch’s picture

Rebased and implemented #8. I'm a bit hazy about whether $this->language is really guaranteed to be the default language in all cases, because otherwise how does ::loadOverrides() work, so it's checking everything it can.

I removed the performance test changes here because they're out of date, seeing what other test failures do or don't happen before worrying about those, but ran one locally and results are similar to #4.

catch’s picture

@me

I'm a bit hazy about whether $this->language is really guaranteed to be the default language in all cases, because otherwise how does ::loadOverrides() work, so it's checking everything it can.

Read a bit more - it's not guaranteed - it can be set dynamically during language negotiation or similar.

Turns out #7 is a 'we were both right' situation - we have to translate English when it's not the default language, and we also have to translate English when locale's 'translate_english' setting is on, even when it's the default language - at least as far as two tests are concerned. I'm not sure whether there's a real use-case for translating English in config when it's the default language - why not update the config instead? But if we wanted to deprecate that or stop it it feels like something to decide independently of trying to speed things up when it's not enabled unless it somehow becomes blocking.

Getting config values within a config overrider is not fun - easy to end up in an infinite loop. I used the raw storage that's already available. Better ideas welcome. However the performance improvement here is dramatic on cold caches and it's also a small improvement on warm/warm-ish caches.

Tests are green now. Needs some extra comments.

catch’s picture

Status: Needs work » Needs review

This works now.

I am thinking though that instead of locale's translate_english setting, we might want to instead add $settings['translate_default_language_config'] - that would avoid getting config in a config overrider, and it would also mean that if the default language is e.g. French we could avoid looking up config overrides in French then.

Not sure if we can get a way with defaulting that to false though, but it seems pretty odd to use config translation to translate config for the default language, so... maybe?

alexpott’s picture

We could do something like we do for the default language and add this to the container - see \Drupal\Core\DrupalKernel::compileContainer() and \Drupal\Core\DrupalKernel::getConfigStorage()... we could do something similar and then inject the container parameter into language.config_factory_override.

In fact I think think we should add something to container so that we can inject any config value into a service by adding something like %config:locale.settings:translate_english%.

catch’s picture

Putting it in the container is a good idea and would prevent the circular dependency. However if I'm right in #13 we'd miss the opportunity to skip override lookups when the default language isn't English. But that might be a follow-up postponed on #2905295: Configuration language being overwritten during module install given one of the proposed solutions in that issue is to essential enforce that the default configuration language remains English whatever happens. If we limit the optimisation to English here the container parameter would be a clean way to do it.

catch’s picture

Two more potential issues, more annoyances than showstoppers:

1. Language module doesn't depend on locale. We can have a soft dependency when the config object doesn't exist but it's a bit smelly.

2. DrupalKernel definitely doesn't care about and shouldn't know about locale, system.site is... questionable but more valid.

Locale module could probably set a container parameter in a service provider, and then language module could look for that container parameter but allow for it not being set. This resolves #2 but not #1. That possibly feels like as far as we should go in this issue, then open follow-ups for the rest.

alexpott’s picture

Locale module could probably set a container parameter in a service provider, and then language module could look for that container parameter but allow for it not being set. This resolves #2 but not #1. That possibly feels like as far as we should go in this issue, then open follow-ups for the rest.

Makes sense to me - plus language module could provide a default that's then changed by the locale module.

catch’s picture

Issue summary: View changes
berdir’s picture

The cache lookup reductions do look nice, to be fair, that's only going to be true for for one language though. I see we have no performance tests yet when accessing a different language, I don't remember if we discussed that before, but that seems like a useful thing to have? In a different issue of course?

catch’s picture

The cache lookup reductions do look nice, to be fair, that's only going to be true for for one language though.

This is true but I think we could theoretically at least not load overrides for whichever language is the default language (on the basis that the configuration will be in that language). But it would be dependent on what happens in #2905295: Configuration language being overwritten during module install and related issues.

A test for a different language sounds good - and easy to add to the Umami tests given it has Spanish available.

catch’s picture

Issue summary: View changes

The container parameter approach saves an extra cache get, not to be sniffed at. Should be ready for review again.

Ran into a weird nightwatch test, turns out that locale.settings had no translate_english value in that test at all, maybe because it's testing the installer? Either way I added a check that the config key actually exists before altering the container. Think it's OK to be defensive there considering it happens within container building.

needs-review-queue-bot’s picture

Status: Needs review » Needs work
StatusFileSize
new91 bytes

The Needs Review Queue Bot tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".

This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

catch’s picture

Status: Needs work » Needs review

Rebased.

smustgrave’s picture

Status: Needs review » Needs work

Know you're working on those tests right now. But seem to be conflicts in the performance tests now.

catch’s picture

Status: Needs work » Needs review

Rebased.

dcam’s picture

I was shocked when I saw how many queries this saves. Well done finding it.

...we also have to translate English when locale's 'translate_english' setting is on, even when it's the default language - at least as far as two tests are concerned.

Thank you for documenting this in a comment because I didn't understand that condition until I read #12.

My main concern is the code duplication from the new conditions in LanguageConfigFactoryOverride, especially given that the translate English condition is a bit unintuitive. Might it be a good idea to move these to a new function on the class that will return a boolean?

Do the changes to LanguageConfigFactoryOverride::loadOverrides() and LanguageConfigFactoryOverride::getOverride() need unit tests?

catch’s picture

Might it be a good idea to move these to a new function on the class that will return a boolean?

Good idea, updated for that.

Do the changes to LanguageConfigFactoryOverride::loadOverrides() and LanguageConfigFactoryOverride::getOverride() need unit tests?

I'm not sure what the existing coverage is - if we already have some then an addition sounds good, not sure I want to write completely new unit tests here given there's lots of kernel/functional testing (as evidenced by earlier test failures).

Rebased for performance test changes too.

dcam’s picture

Status: Needs review » Reviewed & tested by the community

My feedback was addressed. It looks good to me.

longwave’s picture

Version: 11.x-dev » 11.3.x-dev
Status: Reviewed & tested by the community » Fixed

I read through the MR and all comments, the fix makes sense to me and the change in cache get statistics is staggering. If we were earlier in the cycle I would consider asking for backport to 10.x, but at this stage it's likely not worth the effort.

Committed and pushed bd24c65d6e6 to 11.x and a735e53c8a2 to 11.3.x. Thanks!

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.

  • longwave committed a735e53c on 11.3.x
    fix: #3518992 Config overrides are loaded for English even when...

  • longwave committed bd24c65d on 11.x
    fix: #3518992 Config overrides are loaded for English even when...

alexpott’s picture

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

anybody’s picture

Issue summary: View changes
StatusFileSize
new43.94 KB

FYI: We were recently wondering in existing installations that we upgraded to 11.3.2 that English config translations are suddenly broken, see the following screenshot:

This is a regression caused by this issue. The affected projects have been upgraded from Drupal 8 on, so maybe it's an issue with mature Drupal installations, we're further investigating, if it needs to be fixed in the projects or in code here.
In the beginning these projects has "de" (German) as default language. Later on be switched to "en" (English) as default and primary language, because we ran into other core issues. Maybe that caused this edge-case?

I could imagine other people upgradting to 11.3 may also run into this...

If we turn the condition of $this->shouldSkipOverrides() into FALSE, everything works correctly as before the 11.3 upgrade.

anybody’s picture

I think this may happen because in cases like ours for historic reasons default configs (e.g. "sync/system.site.yml" example below) still have langcode "de":

_core:
  default_config_hash: AyT9s8OUcclfALRE_imByOMgtZ19eOlqdF6zI3p7yqo
langcode: de
uuid: 060ad019-4c88-4f3c-97cb-24ac6c5fc5d5
name: 'Drupal 11 CMS Bootstrap'
mail: info@example.com
slogan: 'Das hier ist Deutsch'
page:
  403: ''
  404: /search404
  front: /node/47
admin_compact_mode: false
weight_select_max: 100
default_langcode: en
mail_notification: ''

So it may need an update to correct all the different default configs and turn them into translations...

So I think this case happens, if someone switched the default language over time!

anybody’s picture

The (unwanted and wrong) quick-fix is to set

Enable interface translation to English

TRUE at
/admin/config/regional/language/edit/en so the condition would become FALSE.

But of course that's not what we want and is logically wrong because English is our default language.

So to fix the regression I think we need to update all default configurations to have langcode: en and make these real translations, not defaults.

Furthermore, IMHO I think we could and should simplify all this, by making English the hard-coded default language for Drupal and make everything else a translation, so that for example in config export everything in "sync/" will be en and all other translations go into their dedicated sync/language/XX folder. Database-wise the "collection" column would be empty for these. That would clarify and simplify things a lot logic-wise and code-wise.
I think I already read this idea somewhere in the related issues.

How should we sort out these issues?

Should we open a regression issue for #36 or solve it somewhere else?
I think for our case we'll write an update hook like written above for all legacy default configs that have langcode: de.

anybody’s picture

lendude’s picture

I ran into this too the same issue as @anybody, but I don't think the root cause in our case is a switched default language.

Our case is in Webforms. Webforms are created in either Dutch or English and then translated as needed. The site default language is English. Webforms created in English can be translated to Dutch. After updating to 11.3 Webforms created in Dutch cannot be translated to English anymore and existing translations are not picked up anymore. The proposed fix in #3568743: Convert / unify all default configs to the site default language won't help here since new webforms are created all the time, so running an update won't do the trick.

In our case we needed to switch on 'Enable interface translation to English' in the English language. Which makes sense because that is indeed what we want, but that setting wasn't needed before and is now, so still came as a bit of a surprise and took some debugging to figure out what was going on. So sharing here to maybe save others some time.

anybody’s picture

Thanks @lendude. I think my suggestion to generally store (base) config in english would be the best fix in general, as it would fix all those cases and simplify things a lot for developers and Drupal itself.