Problem/Motivation

If you install Drupal using drush or using #2911319: Provide a single command to install & run Drupal and probably using console too in a language other than English the install_import_translations() step will fail to import translations.

Technical details

On an interactive install the translation project information is changed by _install_prepare_import(). This does not work on CLI installs because locale_translation_get_projects() is statically cached. In locale_translation_build_projects() 8.6.0-dev becomes 8.6.x but install_check_translations() has downloaded a file with the name 8.6.0-dev from the server. Interactive installs use this file because of the way _install_prepare_import() manipulates the project information.

Proposed resolution

Change install_check_translations() to use the same filename as downloading a translation during regular runtime would use. And clear the static cache in _install_prepare_import().

This will also fix:
#2796757: Failing test with stable version string
#2816457: Test fail with stable version
and maybe #2828143: Stop tests like LocaleConfigTranslationImportTest from failing if l.d.o becomes unavailable

Remaining tasks

User interface changes

None

API changes

None

Data model changes

None

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

alexpott created an issue. See original summary.

alexpott’s picture

This patch changes the filename of the downloaded file to drupal-8.6.x.fr.po for both non-interactive and interactive installs. NodeTypeTranslationTest fails now the same way it does for stable releases because non-interactive multi-lingual install works! yay :)

However I think it would be better if the filename did not change.

alexpott’s picture

Status: Needs review » Needs work

The last submitted patch, 3: 2955869-2.patch, failed testing. View results

alexpott’s picture

Status: Needs work » Needs review
FileSize
991 bytes
2.13 KB

Here's a fix for NodeTypeTranslationTest that ensures we don't download from localize.drupal.org

alexpott’s picture

Found a better fix. The problem is occurring because _install_prepare_import() via a browser doesn't have information static cached in locale_translation_get_projects(). Clearing that cache makes CLI and browser install work the same way.

The proof patch is just the fix - which will have NodeTypeTranslationTest broken...

The last submitted patch, 6: 2955869-6.proof_.patch, failed testing. View results

alexpott’s picture

Looking at this a little bit more we can do a better fix since the problem occurs when the project cache get populated during the delete of the english language which happens as part of install_download_additional_translations_operations().

The last submitted patch, 8: 2955869-7.proof_.patch, failed testing. View results

alexpott’s picture

Issue summary: View changes
Issue tags: +Needs manual testing
FileSize
3.08 KB

Okay I finally now understand the problem I think we have two ways to solve it!

1. Clear static cache in _install_prepare_import()

In this method we manipulate the project information so that we use the downloaded translation files. However when installing via CLI the project information is already cached and does not match what we are using here.

2. Make install_check_translations() use the same filename it would do if the language installed later

The problem is occurring because in this early install method we create a file like 8.6.0-dev whereas in locale_translation_build_projects() we change 8.6.0-dev to 8.6.x. We override this information in _install_prepare_import() which makes it work interactively but not in CLI because of the static cache.

I think doing both option 1 and option 2 together is the best way to solve this because it results in the same filename that you'd have if you add the language after installation and the information being set in _install_prepare_import() being used for both CLI and browser install. Therefore merging the fix in #5 in and improving docs. And moving the clear cache to _install_prepare_import().

I have tested this both with drush using the --locale option and the interactive installer. Both result is a site installed in a language other than english being translated. Unfortunately we want tests to not depend on localise.drupal.org so an automated test is not possible. However given that the changes are to a downloaded filename and clearing a static cache I think they are pretty minimal.

alexpott’s picture

Priority: Normal » Major

Making stable and dev versions behave the same is a major bug as is have the test suite pass when the version is stable - see #2796757-8: Failing test with stable version string. As that issue is now postponed on this one marking this issue major.

Sutharsan’s picture

A little historical background will help here. Because the translation server only generates translations files for stable releases, the translation import code was initially only worked with stable releases. At the time, the best thing we could to was to ignore the '-dev' part of a version number. The '-dev' was stripped from the version number. Late in the D8 development (perhaps around 8.0.x-beta) a server side solution for dev releases was introduced. A '.x' translation file is made available (symlinked, if I'm correct) that contains the latest stable translation
For example drupal-8.5.x.nl.po always points to the latest stable (consecutively: drupal-8.5.x-alpha1, drupal-8.5.x-beta1, drupal-8.5.x-rc1 and drupal-8.5.0 (current)).

I have not had the time to look into your solution, but will do in the following days.

dawehner’s picture

  1. +++ b/core/includes/install.core.inc
    @@ -1450,7 +1450,7 @@ function _install_get_version_info($version) {
           \.          # .
    -      (?P<patch>[0-9]+)    # Patch release number.
    +      (?P<patch>[0-9x]+)   # Patch release number.
    
    @@ -1857,10 +1860,19 @@ function install_check_translations($langcode, $server_pattern) {
     
    +  $version = \Drupal::VERSION;
    +  // For dev releases, remove the '-dev' part and trust the translation server
    +  // to fall back to the latest stable release for that branch.
    +  // @see locale_translation_build_projects()
    +  if (preg_match("/^(\d+\.\d+\.).*-dev$/", $version, $matches)) {
    +    // Example match: 8.0.0-dev => 8.0.x (Drupal core)
    +    $version = $matches[1] . 'x';
    +  }
    

    I'm curious, could we add some test coverage for these cases?

  2. +++ b/core/includes/install.core.inc
    @@ -1722,6 +1722,9 @@ function _install_prepare_import($langcodes, $server_pattern) {
               \Drupal::service('locale.project')->set($data['name'], $data);
    ...
    +          // Reset project information static cache so that it uses the data
    +          // set above.
    +          locale_translation_clear_cache_projects();
    

    So this now does the same as what is done at the end of \locale_translation_build_projects which makes sense.

alexpott’s picture

@dawehner test coverage is a bit tricky because we explicit don't want to go l.d.o and we already have test coverage of pre-placing translation files. The bug happens because of different ways of interpreting the fallback solution in place on l.d.o. Basically I'm not sure that the current design of locale in the installer and the installer itself makes this testable.

alexpott’s picture

I worked out how to test this. It's pretty simple really. We can set up the translation files with different strings at the beginning of a new interactive install and have a translation so we can see if it worked or not.

The test only patch is the interdiff.

The last submitted patch, 15: 2955869-15.test-only.patch, failed testing. View results

dawehner’s picture

  1. +++ b/core/modules/locale/tests/src/Functional/LocaleNonInteractiveDevInstallTest.php
    @@ -0,0 +1,21 @@
    +    include_once dirname(dirname(dirname(dirname(dirname(__DIR__))))) . '/includes/install.core.inc';
    
    +++ b/core/modules/locale/tests/src/Functional/LocaleNonInteractiveInstallTest.php
    @@ -0,0 +1,57 @@
    +  protected function getVersionStringToTest() {
    +    include_once dirname(dirname(dirname(dirname(dirname(__DIR__))))) . '/includes/install.core.inc';
    

    Is there a reason we are not just using $this->root ?

  2. +++ b/core/modules/locale/tests/src/Functional/LocaleNonInteractiveInstallTest.php
    @@ -0,0 +1,57 @@
    + * @group locale
    + */
    +class LocaleNonInteractiveInstallTest extends BrowserTestBase {
    +
    

    It is really nice that it is possible to test this¬

alexpott’s picture

Re #17.1 Nope :) fixed.

dawehner’s picture

Status: Needs review » Reviewed & tested by the community

This adds decent amount of test coverage, nice!

alexpott’s picture

@plach pointed out that the regex would match 8.0.0x and 0x would be patch version.

The current _install_get_version_info() actually does nothing to change the validity of the version string but it feels worth it to have a correct regex.

+++ b/core/modules/locale/tests/src/Functional/LocaleNonInteractiveDevInstallTest.php
@@ -0,0 +1,21 @@
+    return $version['major'] . '.' . $version['minor'] . '.x';

+++ b/core/modules/locale/tests/src/Functional/LocaleNonInteractiveInstallTest.php
@@ -0,0 +1,57 @@
+    return $version['major'] . '.0.0';

This gives us coverage of both types of patch - ie. 'x' or '0'

Leaving at rtbc as this is a very minor change that I've tested on regex101.com and with out test coverage. Also note that the only runtime usage is:

preg_match('/drupal-([0-9a-z\.-]+)\.' . $langcode . '\.po/', $filename, $matches);

in _install_prepare_import() where we are working with files downloaded from localize.drupal.org so the version numbers come from there and therefore I guess _install_get_version_info() has been written in way that assumes l.d.o produces valid file names.

alexpott credited plach.

alexpott’s picture

Crediting @plach and @dawehner for reviews.

  • plach committed 4d769bc on 8.6.x
    Issue #2955869 by alexpott, dawehner, plach: Fix multilingual install on...
plach’s picture

Version: 8.6.x-dev » 8.5.x-dev

Committed 4d769bc and pushed to 8.6.x. Thanks!

  • plach committed 493e5fa on 8.5.x
    Issue #2955869 by alexpott, dawehner, plach: Fix multilingual install on...
plach’s picture

Status: Reviewed & tested by the community » Fixed

Committed 493e5fa and pushed to 8.5.x. Thanks!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.