Reproduce

Install Drupal 8.0.0-beta1 in a language which is supported (tested with Norwegian Bokmål - "nb"). Notice that Drupal downloads drupal-7.0.nb.po (into sites/default/files/translations/) instead of drupal-8.0.0-beta1.nb.po - which exists.

Solution

Make the installer semantic versioning aware.

First attempt for a patch is attached. Changes:

1) Added support for patch level in _install_get_version_info
2) Added support for patch level in install_get_localization_release

CommentFileSizeAuthor
#46 interdiff-36-44.txt1.17 KBzaporylie
#44 interdiff-36-44.patch1.17 KBzaporylie
#44 drupal-installer-semantic-versioning-2349263-44.patch13.9 KBzaporylie
#36 interdiff-34-36.txt3.54 KBDésiré
#36 drupal-installer-semantic-versioning-2349263-36.patch13.97 KBDésiré
#34 drupal-installer-semantic-versioning-2349263-33.patch13.96 KBzaporylie
#34 interdiff-2349263-30-33.txt3.33 KBzaporylie
#32 interdiff-2349263-30-31.txt2.79 KBzaporylie
#32 drupal-installer-semantic-versioning-2349263-31.patch13.93 KBzaporylie
#30 drupal-installer-semantic-versioning-2349263-30.patch13.98 KBhansfn
#27 interdiff-25-27.txt1.64 KBDésiré
#27 drupal-installer-semantic-versioning-2349263-27.patch13.55 KBDésiré
#25 interdiff-15-25.txt4.26 KBDésiré
#25 drupal-installer-semantic-versioning-2349263-25.patch13.39 KBDésiré
#15 interdiff-10-15.txt2.71 KBDésiré
#15 drupal-installer-semantic-versioning-2349263-15.patch12.95 KBDésiré
#12 drupal-installer-semantic-versioning-2349263-12.patch34.44 KBhansfn
#11 drupal-installer-semantic-versioning-2349263-11.patch34.44 KBhansfn
#10 interdiff-7-10.txt5.83 KBDésiré
#10 drupal-installer-semantic-versioning-2349263-10.patch34.48 KBDésiré
#7 drupal-installer-semantic-versioning-2349263-7.patch8.71 KBDésiré
#7 interdiff-6-7.txt2.24 KBDésiré
#6 drupal-installer-semantic-versioning-2349263-6.patch8.67 KBhansfn
#4 drupal-installer-semantic-versioning-2349263-2.patch8.67 KBhansfn
#1 drupal-installer-semantic-versioning-2349263.patch6.47 KBhansfn
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

hansfn’s picture

Sutharsan’s picture

Status: Active » Needs work
Issue tags: +D8MI, +language-interface, +Needs manual testing

First pass review. Not yet tested, not checked for missing fallback scenarios due to the extra version level.

  1. +++ b/core/includes/install.core.inc
    @@ -1338,38 +1332,38 @@ function install_get_localization_release($version = \Drupal::VERSION) {
    +      // Dev release: the previous point release (e.g., 8.2.0-dev falls back to
    +      // 8.1.0) or any unstable release (e.g., 8.0.0-dev falls back to 8.0.0-rc1,
    +      // 8.0.0-beta1 or 8.0.0-alpha12)
           case 'dev':
    

    Comment line > 80 chars and not ending with a period.

  2. [edit: rephrased]

  3. +++ b/core/includes/install.core.inc
    @@ -1378,28 +1372,40 @@ function install_get_localization_release($version = \Drupal::VERSION) {
    -      'core' => $core . '.x',
    +      'core' => $core . '.' . $info['minor']. '.x',
           'version' => $alternative,
    

    "core" format does not have a minor. Unchanged at 8.x, etc.

  4. +++ b/core/includes/install.core.inc
    @@ -1378,28 +1372,40 @@ function install_get_localization_release($version = \Drupal::VERSION) {
    +  // All releases may fall back to the previous major release (e.g., 8.1 falls
    +  // back to 7.0, 8.x-dev falls back to 7.0). This will probably only be used
    +  // for early dev releases or languages with an inactive translation team.
    +  if ($info['major'] == 8) {
    

    "8.1" should be "8.1.1". "8.x-dev" should be "8.0.x-dev".

    Add note that this should be revisited in the 9.0.x branch to fall back to 8.0.0.

  5. +++ b/core/includes/install.core.inc
    @@ -1378,28 +1372,40 @@ function install_get_localization_release($version = \Drupal::VERSION) {
    +      'core' => $prev_major . '0.x',
    

    See above.

hansfn’s picture

Thx for the comments. I realize now that core number shouldn't have been changed. I'll try to update the patch later today.

Add note that this should be revisited in the 9.0.x branch to fall back to 8.0.0.

Isn't that already handled by the else part of the following code:

+  if ($info['major'] == 8) {
+    $releases[] = array(
+      'core' => '7.x',
+      'version' => '7.0',
+    );
+  }
+  else {
+    $prev_major = $info['major'] - 1;
+    $releases[] = array(
+      'core' => $prev_major . '0.x',
+      'version' => $prev_major . '0.0',
+    );
+  }
hansfn’s picture

Second attempt attached. This also handles fall back at patch level. It's also commented slightly better - too much?

PS! I have tested my work with the following little file (test.php):

include 'core/includes/install.core.inc';
if (isset($argv[1])) {
  $release = $argv[1];
}
else {
  $release = '8.0.0-beta1';
}
print_r(install_get_localization_release($release));

Running it like "php test.php 8.3.4" and "php test.php 8.0.0-dev".

PS! It seems \Drupal::VERSION is 8.0.0-dev also for the 8.0.0-beta1 release - ref core/lib/Drupal.php Is that correct?

hansfn’s picture

+++ b/core/includes/install.core.inc
@@ -1407,14 +1425,15 @@ function install_get_localization_release($version = \Drupal::VERSION) {
+ *   Version info string (e.g., 8.0.0, 8.1.0, 8.0.0-dev, 8.0.0-unstable1, 8.0.0-alpha2,

OK, another too long comment that I didn't spot. (Testing Dreditor.)

hansfn’s picture

Updated patch with two too long comments fixed.

Désiré’s picture

Status: Needs work » Reviewed & tested by the community
FileSize
2.24 KB
8.71 KB

I'm tested and reviewed the last patch, the fix worked for me, the code seems to be ok.
(My patch only fixes some minor coding style issues, so I put this RTBC.)

Désiré’s picture

Assigned: Unassigned » Désiré
Status: Reviewed & tested by the community » Needs work
Issue tags: +Needs tests

This should have automated tests (or update existing ones).

The last submitted patch, 7: drupal-installer-semantic-versioning-2349263-7.patch, failed testing.

Désiré’s picture

Status: Needs work » Needs review
FileSize
34.48 KB
5.83 KB
  • Updated the test cases.
  • Added new test cases for the point releases.
  • Fixed the logic based on the test changes.
hansfn’s picture

Some minor updates to the fall back detection:

1) Simplified logic since we are already using array_unique to remove duplicates.
2) Fixed fallback for minor > 0, patch = 0 which didn't include 8.0.0.
3) Fixed some comments.

Reviewed code:

1) Not in position to say if the change in core/modules/locale/src/Tests/LocaleUpdateInterfaceTest.php is correct.
2) Don't understand why the changes in core/modules/migrate_drupal/src/Tests/d6/MigrateAggregatorFeedTest.php and core/modules/migrate_drupal/src/Tests/d6/MigrateAggregatorItemTest.php are included in the patch.

Sorry about the missing interdiff.

hansfn’s picture

Arg, some ending white space ...

The last submitted patch, 11: drupal-installer-semantic-versioning-2349263-11.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 12: drupal-installer-semantic-versioning-2349263-12.patch, failed testing.

Désiré’s picture

Status: Needs work » Needs review
FileSize
12.95 KB
2.71 KB

2) Don't understand why the changes in core/modules/migrate_drupal/src/Tests/d6/MigrateAggregatorFeedTest.php and core/modules/migrate_drupal/src/Tests/d6/MigrateAggregatorItemTest.php are included in the patch.

Sorry for that, wrong diff.

Now I'm updated it, included your last changes (from #12), and also changed some comments. The failing test is also fixed.
Interdiff form #10.

1) Not in position to say if the change in core/modules/locale/src/Tests/LocaleUpdateInterfaceTest.php is correct.

They should be correct.
The only question is that we really need fallback for point versions (8.5.4 -> 8.5.3), or it should be enough to fallback to the .0 version (8.5.4 -> 8.5.0)? (Because it's not likely to every point version will have a translation, and even it have after install it can be updated to the freshest easily.)
If we don't need it the code and the test also should be changed.

hansfn’s picture

Status: Needs review » Reviewed & tested by the community

I see that the last patch is less intrusive. It doesn't fix all the update.php to core/update.php things. Probably better - keeping this issue clear.

I have reviewed the latest version of the patch and tested it manually and I think were ready to go. (Remove the testing issue tags?)

The only question is that we really need fallback for point versions (8.5.4 -> 8.5.3), or it should be enough to fallback to the .0 version (8.5.4 -> 8.5.0)?

Yes, we really need that. Imagine when Drupal 8.0.9 is released. If people install it before l.d.o has made a translation package for it, you'll get the translation for 8.0.0 in stead of 8.0.8 - which might be a significant difference. Asking people to redownload the translation immediately after installing isn't user friendly.

My only question is: Isn't there rc/beta/alpha releases for 8.x.0 when X > 0? I mean 8.0.0-dev falls back to 8.0.0-rc1, 8.0.0-beta1, 8.0.0-alpha1 and 7.0 while 8.2.0-dev falls back to 8.1.0 and 7.0. I don't know how the minor release are planned so I can't tell.

The last submitted patch, 6: drupal-installer-semantic-versioning-2349263-6.patch, failed testing.

The last submitted patch, 4: drupal-installer-semantic-versioning-2349263-2.patch, failed testing.

The last submitted patch, 1: drupal-installer-semantic-versioning-2349263.patch, failed testing.

Sutharsan’s picture

Status: Reviewed & tested by the community » Needs work
Issue tags: -Needs manual testing, -Needs tests
  • +++ b/core/includes/install.core.inc
    @@ -1308,90 +1307,103 @@ function install_check_localization_server($uri) {
    +  // wasn't introduse before "alpha13". However, we have already gotten to
    

    Typo. Change "introduse" to "introduced".

  • +++ b/core/includes/install.core.inc
    @@ -1308,90 +1307,103 @@ function install_check_localization_server($uri) {
    +    // Below we assume that for all dev, alpha, beta or rc releases
    +    // the patch level is 0. Modify code below and include $info['patch']
    +    // if the assumption is wrong.
    

    I have no indication that there will releases like 8.1.1-rc1. Let's base the code on what we currently know and modify it when new information comes up.

  • +++ b/core/includes/install.core.inc
    @@ -1308,90 +1307,103 @@ function install_check_localization_server($uri) {
    +      // Dev release: The previous point release (e.g., 8.2.0-dev falls back
    

    Comment line too short.

  • +++ b/core/includes/install.core.inc
    @@ -1308,90 +1307,103 @@ function install_check_localization_server($uri) {
    +      // Dev release: The previous point release (e.g., 8.2.0-dev falls back
    +      // to 8.1.0) or any unstable release (e.g., 8.0.0-dev falls back to
    +      // 8.0.0-rc1, 8.0.0-beta1 or 8.0.0-alpha1).
           case 'dev':
             if ($info['minor'] >= 1) {
    -          $alternatives[] = $info['major'] . '.' . ($info['minor'] - 1);
    +          $alternatives[] = $info['major'] . '.' . ($info['minor'] - 1) . '.0';
             }
             else {
    

    This also means that for example 8.0.8-dev falls back to 8.0.0. There may be several months between these releases. Let's add a fall back to 8.0.8 too.

  • Asking people to redownload the translation immediately after installing isn't user friendly.

    There is always room for improvement. I suggest to create a follow-up for this.

    My only question is: Isn't there rc/beta/alpha releases for 8.x.0 when X > 0?

    I have no information that we will. We can fix it when it comes up, also in a separate issue.

    Removing test tags as we have now both manual and automated test coverage.

    zaporylie’s picture

    Priority: Normal » Major
    Issue tags: +Needs manual testing, +Needs tests

    I have small digression into issue priority. I've bumped it up, because I think this is a really major bug. Right now, installator takes really old .po file, with really outdated strings, related to many other, already fixed issues (like #1317884: Remove all instances of <none>, <Hidden> and <br/> from translatable strings because they lead to import errors). I was so confused about it so I started digging into installator and found the reason - missing support for semantic versioning.
    Conclusion:
    This issue should be fixed and committed asap.

    zaporylie’s picture

    Sutharsan’s picture

    Just talked to Alex Pott on future version numbers. We should anticipate on versions like "8.1.0-rc1"

    Sutharsan’s picture

    Assigned: Désiré » Sutharsan
    Désiré’s picture

    Assigned: Sutharsan » Désiré
    Status: Needs work » Needs review
    FileSize
    13.39 KB
    4.26 KB
    • Fixed everything from #20.
    • Changed behavior for rc point releases (#23)
    • The fallback chain for 8.2.0-rc2 is '8.2.0-rc2', '8.2.0-rc1', '8.1.0', '8.0.0', '7.0'. Discussed it with @Sutharsan, '8.1.0' is better there, but not necessary, but the code is also less complicated this way.

    (I don't know how important or how frequently is this code changing, but it's very ugly and hard to work on it, probably it should be completely refactored in an other issue.)

    Sutharsan’s picture

    Further discussed future release versions with Alex Pott in IRC:
    alexpott: You confirmed that we may expect 8.1.0-rc* releases. What about beta/alpha for 8.1.0 and onwards? e.g. 8.1.0-beta1 (c.c. DesireJM)
    Sutharsan: we'll talk about this in between the committers

    Lets see if we can keep this option open in the code, so we do not need to change it again when is gets decided.

    Désiré’s picture

    I'm added the fallback for 8.x.0-rc -> 8.x.0-beta1.

    There are also some other inconsistent fallback chains, like:

    • '8.0.0-beta2', '8.0.0-beta1', '7.0' should include an alpha fallback
    • 8.9.0-dev falls back to '8.8.0', '7.0', while at least 8.0.0 should be included there

    But the current code is really hard to change, and any new rule could overcomplicate it. So at this point I suggest to commit this when it's possible since the original bug is fixed, and submit a followup issue for make more consistent fallback chains and a refactor for the logic.

    Sutharsan’s picture

    Status: Needs review » Needs work
    +++ b/core/includes/install.core.inc
    @@ -1396,6 +1396,7 @@ function install_get_localization_release($version = \Drupal::VERSION) {
             // Minor releases can have rc releases, but not beta or alpha.
             if ($info['minor'] > 0) {
    +          $alternatives[] = $info['major'] . '.' . ($info['minor']) . '.0-beta1';
               $alternatives[] = $info['major'] . '.' . ($info['minor'] - 1) . '.0';
               $alternatives[] = $info['major'] . '.0.0';
    

    Comment needs to be updated

    +++ b/core/modules/system/src/Tests/Installer/InstallerTranslationVersionUnitTest.php
    @@ -62,56 +62,83 @@ protected function assertVersionFallback($version, $fallback, $message = '', $gr
    -    $version = '8.0-beta1';
    -    $fallback = array('8.0-beta1', '8.0-alpha2', '7.0');
    +    $version = '8.0.0-beta1';
    +    $fallback = array('8.0.0-beta1', '8.0.0-alpha1', '7.0');
    

    Add test cases for 8.1.0-beta1 and -beta*

    Sutharsan’s picture

    Discussed with Désiré to finalise this issue to handle semantic 8.x.0-dev/beta/alpha releases too. Désiré has ideas for refactoring the code, but we will do that in a separate issue.

    hansfn’s picture

    Another attempt from me. Fixes everything mentioned above and slightly refactored - maybe for the better. All tests still pass.

    zaporylie’s picture

    Status: Needs work » Needs review

    Queued for automated testing.

    zaporylie’s picture

    @hansfn - I really like your work! Thanks!
    Anyway, I've added a few changes to cover more cases, removed trailing whitespace in tests file, added a sentence to explain that duplicated alternatives will be removed at the end of process.
    New case covered:

    • added rc1 as a fallback to current minor version with patch == 0 (although I'm pretty sure that rc3 and rc2 are even more important and should be taken before rc1)

    Status: Needs review » Needs work

    The last submitted patch, 32: drupal-installer-semantic-versioning-2349263-31.patch, failed testing.

    zaporylie’s picture

    Status: Needs work » Needs review
    FileSize
    3.33 KB
    13.96 KB

    I've forgotten about tests

    Sutharsan’s picture

    added rc1 as a fallback to current minor version with patch == 0 (although I'm pretty sure that rc3 and rc2 are even more important and should be taken before rc1)

    It was never the intention of this hardcoded version fallback to be complete. It is easy to fall back to the first (rc1, beta1), only with an up to date list of releases (like Update module uses) is needed to get the most recent release. The latter is way out of scope of this issue.

    Désiré’s picture

    I'm like #34, it simplifies the logic, and introducing reasonable fallback chains, it also introduce the suggested change from #29. So I'm suggesting RTBC here.

    In this patch I'm just changing some coding standard problems (long arrays are in one line again just to match the style in the same file), and adding only one logic change: 8.1.0-foo1 will now fallback to 8.1.0 before 7.0 because it's not to hard to do, while in the translation versions will be huge differences.

    zaporylie’s picture

    8.1.0-foo1 will now fallback to 8.1.0 before 7.0

    I'm not sure if this fallback is necessary because every 8.x.0-foo1 should be released before 8.x.0.

    Désiré’s picture

    I'm not sure if this fallback is necessary because every 8.x.0-foo1 should be released before 8.x.0.

    No, it's an other case: valid extra names (dev, alpha, beta, rc) should be released before the final version release, and all these cases are handled. But 'foo1' here is an example for an invalid extra name, which should not exists. So if something like this happens we should do something, and I think it's better to fallback some fresher version than only to 7.0.

    zaporylie’s picture

    You are absolutely right that we should do something, so...

    +++ b/core/modules/system/src/Tests/Installer/InstallerTranslationVersionUnitTest.php
    @@ -62,56 +62,84 @@ protected function assertVersionFallback($version, $fallback, $message = '', $gr
    +    $version = '8.0.0-foo2';
    +    $fallback = array('8.0.0', '7.0');
    +    $this->assertVersionFallback($version, $fallback);
    

    ...so probably we should extend Test above, by adding another one for version 8.3.0-foo2, with fallback path: 8.3.0, 8.2.0, 8.0.0, 7.0 what is covered by your last patch, or extend fallbacks list of fake version with three steps more: 8.3.0, 8.3.0-rc1, 8.3.0-beta1, 8.3.0-alpha1, 8.2.0, 8.0.0, 7.0.
    Fallbacks 8.2.0 and 8.0.0 was already covered by #30:

    +++ b/core/includes/install.core.inc
    @@ -1308,90 +1307,87 @@ function install_check_localization_server($uri) {
    +    // All unstable releases except the zero release: The previous minor release
    +    // and the zero release. Duplicates will be removed later.
    +    if ($info['minor'] > 0) {
    +      $alternatives[] = $info['major'] . '.' . ($info['minor'] - 1) . '.0';
    +      $alternatives[] = $info['major'] . '.0.0';
         }
    

    What do you think?

    hansfn’s picture

    Status: Needs review » Reviewed & tested by the community

    OK, I have tested and reviewed - again. Some comments:

    1) The addition of the two "Duplicates will be removed later." strings doesn't add anything. There is already the string "The fallback detection is relaxed - removing version duplicates at the end." at the top, and if we really want to use "Duplicates will be removed later." everywhere it needs to be added also for the Patch-level comment.

    2) Do we really want to spend any more time on invalid releases (foowhatever)? The current fallback is more than sufficient IMHO.

    Désiré’s picture

    Assigned: Désiré » Unassigned
    hansfn’s picture

    I notice that Désiré has unassigned himself/herself. Are we just waiting for this to be committed or is there anything else that needs to be done?

    zaporylie’s picture

    Assigned: Unassigned » zaporylie
    Status: Reviewed & tested by the community » Needs work

    Later today I'll remove "Duplicates will be removed later" from comments (somehow I've missed "The fallback detection is relaxed..." string), than patch will be ready to commit.

    zaporylie’s picture

    Assigned: zaporylie » Unassigned
    Status: Needs work » Reviewed & tested by the community
    Issue tags: +Needs committer feedback
    FileSize
    13.9 KB
    1.17 KB

    Ok, here's patch and interdiff

    Status: Reviewed & tested by the community » Needs work

    The last submitted patch, 44: interdiff-36-44.patch, failed testing.

    zaporylie’s picture

    Status: Needs work » Reviewed & tested by the community
    FileSize
    1.17 KB

    Here is interdiff with right, .txt extension.

    alexpott’s picture

    Status: Reviewed & tested by the community » Fixed

    Committed 9deda26 and pushed to 8.0.x. Thanks!

    • alexpott committed 9deda26 on 8.0.x
      Issue #2349263 by Désiré, zaporylie, hansfn: Fixed Add support for...

    Status: Fixed » Closed (fixed)

    Automatically closed - issue fixed for 2 weeks with no activity.

    xjm’s picture

    Issue tags: -Needs committer feedback