Problem/Motivation

@phenaproxima and I found this while working on #3605554: Translations are never loaded or downloaded for a custom profile. Drupal core's version on the main branch is 12.0-dev. In install.core.inc Drupal attempts to download https://ftp.drupal.org/files/translations/all/drupal/drupal-12.0-dev.de.po (as an example in German) which falls back / gets redirected on the server side to https://ftp.drupal.org/files/translations/all/drupal/drupal-12.x.de.po. But that does not exist or work with fallbacks until the first alpha release of Drupal 12 is tagged. Localize does not know a "12 branch" exists for Drupal until that happens.

Steps to reproduce

Proposed resolution

To make foreign language install work on 'main' we would either need a core built-in fallback mechanism or a server side fallback mechanism. Given the other fallbacks are implemented on the server side, we propose to implement this too there. See comment #3 for proposed resolution.

Remaining tasks

User interface changes

Introduced terminology

API changes

Data model changes

Release notes snippet

Issue fork drupal-3605895

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

gábor hojtsy created an issue. See original summary.

gábor hojtsy’s picture

gábor hojtsy
he/him
Primary language Hungarian
Location Hungary
commented

@Berdir pointed me to #3594389: Remove unecessary core dev version replacement in locale project info which already deals with the -dev removal not being needed. Then it is a lack of fallback that remains. @Berdir also suggested the fallback happens on ftp.drupal.org which also makes sense. Will need to rewrite this issue and move this to the infra queue then :)

alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
commented

The current nginx rules are:

server {
  listen 8080;
  server_name  localize-legacy-static.*;

  # https://docs.fastly.com/guides/tutorials/cache-control-tutorial
  add_header Surrogate-Control "max-age=31536000";
  index index.html;
  autoindex on;
  allow all;

  set_real_ip_from 0.0.0.0/0;
  real_ip_header True-Client-IP;

  location / {
    root /app/web;
    location /healthcheck.txt {
      access_log off;
    }
  }
  location /files/translations {
    root /app/web;
    try_files $uri $uri/ @rewrite;
  }
  location @rewrite {
    root /app/web;

    # This redirects non-existant translations to the "latest" existing translation. For example:
    # https://ftp.drupal.org/files/translations/7.x/pathauto_persist/pathauto_persist-7.x-1.3.de.po
    # doesn't exist on the file system, so try_files will get to @rewrite, and match returning a 302 for:
    # https://ftp.drupal.org/files/translations/7.x/pathauto_persist/pathauto_persist-7.x-1.x.de.po
    # which may or may not exist.
    # Another example is:
    # https://ftp.drupal.org/files/translations/8.x/drupal/drupal-8.0.0-beta20.ca.po
    # which returns
    # https://ftp.drupal.org/files/translations/8.x/drupal/drupal-8.0.x.ca.po
    # and this file exists so it can be downloaded.
    #
    # See ("Rely on proper server side version fallback for translations")
    # https://www.drupal.org/node/2113955 and related issues for more details.
    rewrite '^/files/translations/(.+)/(.+)/([^-]+)-([0-9]+\.([0-9]+\.)?)[0-9]+.+\.([a-z-]+)\.po$'
      https://$host/files/translations/$1/$2/$3-$4x.$6.po;
      #https://ftp.drupal.org/files/translations/$1/$2/$3-$4x.$6.po;
    rewrite '^/files/translations/(.+)/(.+)/(.+)-([0-9]\.([0-9]\.)?x)-([0-9]+)\.[0-9].+\.([a-z-]+)\.po$'
      https://$host/files/translations/$1/$2/$3-$4-$6.x.$7.po;
      #https://ftp.drupal.org/files/translations/$1/$2/$3-$4-$6.x.$7.po;
    return 404;
  }
}

Thanks @drumm for providing them.

I propose that we update them to support getting the previous major version for Drupal core. I considered generalising this for all contrib but it's harder to make assurances about contrib major versions and it does not have as dramatic an impact as core.

http {
  map $major $prev_major {
      default   "";
      12  11;
      13  12;
      14  13;
      15  14;
      16  15;
      17  16;
      18  17;
      19  18;
      20  19;
      21  20;
      22  21;
      23  22;
      24  23;
  }

  server {
    listen 8080;
    server_name  localize-legacy-static.*;

    # https://docs.fastly.com/guides/tutorials/cache-control-tutorial
    add_header Surrogate-Control "max-age=31536000";
    index index.html;
    autoindex on;
    allow all;

    set_real_ip_from 0.0.0.0/0;
    real_ip_header True-Client-IP;

    location / {
      root /app/web;
      location /healthcheck.txt {
        access_log off;
      }
    }

    # Drupal core only: any major .x translation file that doesn't exist 
    # on disk falls back to the previous major's file.
    # e.g. drupal-12.x.fr.po missing -> redirect to drupal-11.x.fr.po
    location ~ ^/files/translations/all/drupal/(drupal-([0-9]+)\.x\.([a-z-]+)\.po)$ {
      root /app/web;
      set $major      $2;
      set $lang       $3;
      try_files $uri @drupal_major_fallback;
    }

    location @drupal_major_fallback {
      if ($prev_major = "") { return 404; }
      return 302 https://$host/files/translations/all/drupal/drupal-$prev_major.x.$lang.po;
    }

    location /files/translations {
      root /app/web;
      try_files $uri $uri/ @rewrite;
    }
    location @rewrite {
      root /app/web;

      # This redirects non-existant translations to the "latest" existing translation. For example:
      # https://ftp.drupal.org/files/translations/7.x/pathauto_persist/pathauto_persist-7.x-1.3.de.po
      # doesn't exist on the file system, so try_files will get to @rewrite, and match returning a 302 for:
      # https://ftp.drupal.org/files/translations/7.x/pathauto_persist/pathauto_persist-7.x-1.x.de.po
      # which may or may not exist.
      # Another example is:
      # https://ftp.drupal.org/files/translations/8.x/drupal/drupal-8.0.0-beta20.ca.po
      # which returns
      # https://ftp.drupal.org/files/translations/8.x/drupal/drupal-8.0.x.ca.po
      # and this file exists so it can be downloaded.
      #
      # See ("Rely on proper server side version fallback for translations")
      # https://www.drupal.org/node/2113955 and related issues for more details.
      rewrite '^/files/translations/(.+)/(.+)/([^-]+)-([0-9]+\.([0-9]+\.)?)[0-9]+.+\.([a-z-]+)\.po$'
        https://$host/files/translations/$1/$2/$3-$4x.$6.po;
      rewrite '^/files/translations/(.+)/(.+)/(.+)-([0-9]\.([0-9]\.)?x)-([0-9]+)\.[0-9].+\.([a-z-]+)\.po$'
        https://$host/files/translations/$1/$2/$3-$4-$6.x.$7.po;
      return 404;
    }
  }
}

LLM disclaimer: I worked with an LLM to update the ruleset to support this new fallback behaviour.

In order to test this we need to confirm that:

  • https://ftp.drupal.org/files/translations/all/drupal/drupal-12.0-dev.fr.po falls back to https://ftp.drupal.org/files/translations/all/drupal/drupal-12.x.fr.po and then https://ftp.drupal.org/files/translations/all/drupal/drupal-11.x.fr.po and successfully downloads a file.
  • https://ftp.drupal.org/files/translations/all/drupal/drupal-11.4-dev.fr.po falls back to https://ftp.drupal.org/files/translations/all/drupal/drupal-11.x.fr.po (current behaviour)
  • https://ftp.drupal.org/files/translations/all/drupal/drupal-13.0-dev.fr.po falls back to https://ftp.drupal.org/files/translations/all/drupal/drupal-13.x.fr.po and then https://ftp.drupal.org/files/translations/all/drupal/drupal-12.x.fr.po and then https://ftp.drupal.org/files/translations/all/drupal/drupal-11.x.fr.po and successfully downloads a file.
  • https://ftp.drupal.org/files/translations/all/drupal/drupal-25.0-dev.fr.po should 404
  • https://ftp.drupal.org/files/translations/all/drupal/drupal-13.0-dev.xxxxx.po falls back to https://ftp.drupal.org/files/translations/all/drupal/drupal-13.x.xxxxx.po and then https://ftp.drupal.org/files/translations/all/drupal/drupal-12.x.xxxxx.po and then https://ftp.drupal.org/files/translations/all/drupal/drupal-11.x.xxxxx.po and then 404s.
alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
commented
Project: Drupal core » localize.drupal.org
Version: main » 2.x-dev
Component: install system » New language request

Moving to the localize.drupal.org project as this fix now involves server config for it.

alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
commented
Status: Active » Needs review

According to @drumm this can be implemented once localize.drupal.org has moved to AWS which will hopefully happen in the next couple of weeks.

gábor hojtsy’s picture

gábor hojtsy
he/him
Primary language Hungarian
Location Hungary
commented
Title: The main branch does not have corresponding translations on localize.drupal.org, fall back one major version » Drupal core main branch does not have corresponding .po files on localize.drupal.org until first alpha is tagged, fall back one major version
Issue summary: View changes

Making it clear this is about Drupal core and under what condition in the title. Also updated issue summary.