Problem/Motivation

Follow-up for #3008025: Migrate D7 i18n block translated strings.

d7_block_translation does not depend on language. But d6_custom_block_translation, d7_menu_links_translation, d7_language_negotiation_settings and so many others do.

So shouldn't d7_block_translation too?

Steps to reproduce

Proposed resolution

Update the migrations and the Kernel tests.
Further tests will not be needed. In this case, regression is extremely unlikely, because removing a dependency from a migration will ring alarm bells.

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

CommentFileSizeAuthor
#58 3189463-D89.patch24.4 KBGyD
#51 3189463-51.patch33.14 KBquietone
#51 diff-44-51.txt2.67 KBquietone
#44 3189463-44.patch35.68 KBquietone
#44 interdiff-41-44.txt479 bytesquietone
#42 3189463-41.patch35 KBquietone
#42 interdiff-39-41.txt400 bytesquietone
#39 3189463-39.patch35.39 KBquietone
#39 interdiff-34-39.txt2.01 KBquietone
#36 3189463-34-9.1.7.patch33.98 KBWim Leers
#34 3189463-34.patch33.98 KBquietone
#34 interdff-32-34.txt1.24 KBquietone
#32 3189463-32.patch32.73 KBquietone
#32 interdiff-31-32.txt734 bytesquietone
#31 3189463-31.patch31.66 KBquietone
#31 interdiff-30-31.txt2.9 KBquietone
#30 3189463-30.patch28.76 KBquietone
#30 interdiff-25-30.txt2.81 KBquietone
#28 3189463-25-D7-fixes-only-9.1.3.patch9.64 KBWim Leers
#25 3189463-25.patch25.95 KBquietone
#25 interdiff-22-25.txt8.09 KBquietone
#24 3189463-22-fix-only.patch9.21 KBWim Leers
#22 3189463-22.patch14.71 KBquietone
#22 diff-14-22.txt2.69 KBquietone
#17 diff_8-14.patch2.1 KBadityasingh
#14 3189463-14.patch14.86 KBayushmishra206
#8 3189463-8.patch17.33 KBWim Leers
#8 interdiff.txt6.04 KBWim Leers
#5 3189463-4.patch9.88 KBWim Leers
#3 3189463-3.patch9.68 KBWim Leers
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Wim Leers created an issue. See original summary.

quietone’s picture

Arg, probably worth double checking all the translation migrations.

Wim Leers’s picture

Title: Shouldn't the d7_block_translation migration depend on the language migration? » all translation/localization migrations should depend on the 'language' migration
Priority: Minor » Normal
Status: Active » Needs review
Issue tags: +migrate-d7-d8, +multilingual
FileSize
9.68 KB

Aight, that's the nudge I needed 🤓

AFAICT there's even more of these…

Let's see what happens if I make all Drupal 7-tagged migrations in the content_translation, config_translation and language modules depend on language

quietone’s picture

@Wim Leers, I see that I got lax on the dependencies on the translation migrations. Thanks for picking up the slack.

I count 43 migrations with 'translation' in the title and only 11 have 'language' as a dependency, leaving 32 to change. I didn't expect it to be that many.

Wim Leers’s picture

Wim Leers’s picture

I count 43 migrations with 'translation' in the title and only 11 have 'language' as a dependency, leaving 32 to change. I didn't expect it to be that many.

#5 updates 19 of them!

Status: Needs review » Needs work

The last submitted patch, 5: 3189463-4.patch, failed testing. View results

Wim Leers’s picture

Wim Leers’s picture

I think #5 & #8 update all the D7 cases. I did not touch any of the D6 ones.

Status: Needs review » Needs work

The last submitted patch, 8: 3189463-8.patch, failed testing. View results

Wim Leers’s picture

Status: Needs work » Needs review

Random fail in Drupal\Tests\media_library\FunctionalJavascript\WidgetUploadTest. Retesting.

quietone’s picture

Issue summary: View changes
Status: Needs review » Needs work
Issue tags: +i18n-migrate, +Needs reroll, +Novice

Finally got back to review this. And it needs a reroll! This reroll is suitable for a novice.

ayushmishra206’s picture

Assigned: Unassigned » ayushmishra206
ayushmishra206’s picture

Assigned: ayushmishra206 » Unassigned
Status: Needs work » Needs review
Issue tags: -Needs reroll, -Novice
FileSize
14.86 KB

Rerolled the patch as suggested. Please review.

Status: Needs review » Needs work

The last submitted patch, 14: 3189463-14.patch, failed testing. View results

quietone’s picture

@ayushmishra206, can you add the diff?

adityasingh’s picture

Adding the diff.

adityasingh’s picture

oops, #17 saved with wrong extension.

quietone’s picture

@adityasingh, Thanks you for the diff. It does show the mistakes in the reroll, which I have identified below. They should be easy to fix. Obviously, I don't know how you are doing reroll but using the method in Rerolling patches should help prevent these errors. Cheers,

Looking at the diff

  1. +++ 3189463-14.patch	2021-01-29 14:01:28.786759376 +0530
    @@ -175,18 +147,6 @@
    -diff --git a/core/modules/content_translation/migrations/d7_block_translation.yml b/core/modules/content_translation/migrations/d7_block_translation.yml
    

    The changes to this file need to be restored.

  2. +++ 3189463-14.patch	2021-01-29 14:01:28.786759376 +0530
    @@ -327,7 +287,7 @@
    -+      'd7_language_content_settings',
    ++      'd7_language_content_comment_settings',
    

    The migration should not be changed here.

adityasingh’s picture

@quietone

I don't know how you are doing reroll

I haven't re-rolled the patch.

I only shared the diff requested in #16, to help review the patch in #14. I appreciate your consistency, but we should be more vigilant while being suggestive / critical.

quietone’s picture

@adityasingh, my sincere apologies for that mistake. Thank you for pointing out my error.

quietone’s picture

Status: Needs work » Needs review
FileSize
2.69 KB
14.71 KB

I wanted to use this patch for some testing with #3191782: Fix dependency in d6 user profile translation migrations in order to tracking down the problem here, #3191782: Fix dependency in d6 user profile translation migrations. It needed a reroll, so I have done that.

Status: Needs review » Needs work

The last submitted patch, 22: 3189463-22.patch, failed testing. View results

Wim Leers’s picture

FileSize
9.21 KB

I wonder why #22 is failing on MigrateLanguageContentSettingsTest, since it passed just fine in #8 🤔

For migrations that are currently being worked on, here's a fix-only patch (without the test changes), which should be possible to apply to both 9.1.x and 9.2.x. That should help migration projects until this patch lands.

quietone’s picture

24. It is failing due to a mistake in the reroll.

19.1 Another mistake. d7_block_translation should have a dependency on language
19.2. Fixed

This also adds the d6 migrations.

quietone’s picture

Status: Needs work » Needs review
Wim Leers’s picture

#25: D'oh, you just beat me to it!

19.1 Another mistake. d7_block_translation should have a dependency on language

I just noticed the same!

Wim Leers’s picture

For migrations that are currently being worked on, here's a D7-fixes-only patch (without the test or the D6 migration changes), which applies to 9.1.3 & 9.1.4— for the upcoming 9.1.5 release, #3187415: Module settings translation migrations should depend on the default settings migration has been applied and has moved a migration to a different location, hence the need for this patch.

The last submitted patch, 25: 3189463-25.patch, failed testing. View results

quietone’s picture

And again, I forgot to update the Migration tests.

quietone’s picture

A few more migrations.

According to the script below that leaves the entity translation migrations.
< core/modules/content_translation/migrations/d7_entity_translation_settings.yml
30d28
< core/modules/content_translation/migrations/d7_entity_reference_translation.yml
41d38
< core/modules/content_translation/migrations/d6_entity_reference_translation.yml

#!/usr/bin/env bash

find core/modules/*/migrations -iname \*translation\* -not -path "*/state/*" > migrations.out
find core/modules/*/migrations -iname \*translation\* -not -path "*/state/*"| xargs grep -A20 migration_dependencies | grep ' - language' | awk -F- '{print $1}' > migrations_with_language_dependency.out
diff migrations.out migrations_with_language_dependency.out
quietone’s picture

And the entity reference translation migrations.

That just leaves core/modules/content_translation/migrations/d7_entity_translation_settings.yml which I don't think needs the language migration.

Wim Leers’s picture

#31: woah, nice script-fu! 😄

That just leaves core/modules/content_translation/migrations/d7_entity_translation_settings.yml which I don't think needs the language migration.

Hm, why not that one, if we do add the dependency for core/modules/language/migrations/d7_language_content_settings.yml and core/modules/content_translation/migrations/d7_language_content_comment_settings.yml? 🤔

quietone’s picture

@Wim Leers, thank you.

Hm, why not that one,

No clue. I must have been off with the fairies (day dreaming).

Fixed now.

huzooka’s picture

Wim Leers’s picture

Port of #34 to 9.1.7.

Wim Leers’s picture

Status: Needs review » Reviewed & tested by the community

Also in manual testing this works great 👍😊

Thanks for pushing this across the finish line, @quietone! 🙏

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

quietone’s picture

Status: Reviewed & tested by the community » Needs review
FileSize
2.01 KB
35.39 KB

This needed a reroll because d6 and d7 block_translation moved to config_translation. I also added language as a dependency to a couple more migration; d6_language_content_comment_settings, d6_language_types and language.

Status: Needs review » Needs work

The last submitted patch, 39: 3189463-39.patch, failed testing. View results

Wim Leers’s picture

#39:

This needed a reroll because d6 and d7 block_translation moved to config_translation

Correct, that already was the case in your #34 patch for 9.2 :)

I should've made it more clear in #36 probably that that was literally the only change I made between #34 (for 9.2) and #36 (for 9.1). I just wanted to be able to apply this excellent patch to the current stable release!

I also added language as a dependency to a couple more migration; d6_language_content_comment_settings, d6_language_types and language.

Oh, nice catch! 😄 Unfortunately it looks like that broke the tests.

+++ b/core/modules/language/migrations/language.yml
@@ -18,3 +18,6 @@ process:
+migration_dependencies:
+  required:
+    - language

I bet this caused it … this is making language depend on itself!

While this is nonsensical, it shouldn't crash the system.

Instead, we should make the migration definition parser throw an explicit \LogicException. That would've prevented these mind-bendingly annoyingly useless error messages:

 PHP Fatal error:  Uncaught Exception: Serialization of 'Closure' is not allowed in Standard input code:89
Stack trace:

… although we should fix those too in #3197324: Exception trace cannot be serialized because of closure.

quietone’s picture

Status: Needs work » Needs review
FileSize
400 bytes
35 KB

Changed one to many migrations.

Status: Needs review » Needs work

The last submitted patch, 42: 3189463-41.patch, failed testing. View results

quietone’s picture

Status: Needs work » Needs review
FileSize
479 bytes
35.68 KB

Need to add the language migration to a test.

@Wim Leers, "mind-bendingly annoyingly useless error messages" is a far more polite expression that the one I use when I get that error. I waste a lot of time because of #3197324: Exception trace cannot be serialized because of closure.

Wim Leers’s picture

Status: Needs review » Reviewed & tested by the community

is a far more polite expression that the one I use when I get that error.

😅😂😬

I waste a lot of time because of […]

I'll try to push that forward — if it saves you time it is 💯worth my time!


Back to RTBC — this looks great once again! 👍

huzooka’s picture

If you know where the test fails, then something like this helps:

+    try {
+      $this->executeMediaMigrations();
+    }
+    catch (\Exception $exception) {
+      throw new AssertionFailedError(
+        $exception->getMessage() . "\n" .
+        $exception->getLine() . "\n" .
+        $exception->getTraceAsString()
+      );
+    }

(Copied from Media Migration)

larowlan’s picture

Status: Reviewed & tested by the community » Needs review

Playing devil's advocate, how can we be sure that we don't inadvertently add another migration that should depend on language but does not.
Is there a way we can add a test to prevent it sneaking back.

Or in other words, can we automate some of quietone's know-how - specifically

I count 43 migrations with 'translation' in the title and only 11 have 'language' as a dependency, leaving 32 to change.

quietone’s picture

First, this happened because I simply forgot to go back and set the dependencies on all the migrations I wrote. Somehow that step never got into my workflow. :-(

After an admittedly brief time thinking about how to test for this nothing popped up. Figuring out the dependencies is a matter of experience and what is learned when writing the migration kernel test. Plus, I am not highly motivated to think about it because as far as I know all the necessary migrations have been added to core. It is very unlikely that a new one will be added; effort is better spent on other issues in the migration system.

Does that help?

larowlan’s picture

Status: Needs review » Reviewed & tested by the community

Plus, I am not highly motivated to think about it because as far as I know all the necessary migrations have been added to core. It is very unlikely that a new one will be added; effort is better spent on other issues in the migration system.

Perfect, that's an excellent point

larowlan’s picture

Status: Reviewed & tested by the community » Needs work
Issue tags: +Needs re-roll
error: patch failed: core/modules/config_translation/migrations/d6_profile_field_option_translation.yml:49
error: core/modules/config_translation/migrations/d6_profile_field_option_translation.yml: patch does not apply
error: patch failed: core/modules/config_translation/migrations/d6_user_profile_field_instance_translation.yml:40
error: core/modules/config_translation/migrations/d6_user_profile_field_instance_translation.yml: patch does not apply
error: patch failed: core/modules/config_translation/tests/src/Kernel/Migrate/d6/MigrateUserProfileFieldInstanceTranslationTest.php:26
error: core/modules/config_translation/tests/src/Kernel/Migrate/d6/MigrateUserProfileFieldInstanceTranslationTest.php: patch does not apply
error: patch failed: core/modules/user/tests/src/Kernel/Migrate/d6/MigrateProfileFieldOptionTranslationTest.php:29
error: core/modules/user/tests/src/Kernel/Migrate/d6/MigrateProfileFieldOptionTranslationTest.php: patch does not apply

I tried to commit this, but it no longer applies

quietone’s picture

Status: Needs work » Needs review
Issue tags: -Needs re-roll
FileSize
2.67 KB
33.14 KB

Status: Needs review » Needs work

The last submitted patch, 51: 3189463-51.patch, failed testing. View results

quietone’s picture

Status: Needs work » Needs review

Known random failure, Drupal\BuildTests\Framework\Tests\BuildTestTest::testPortMany

Wim Leers’s picture

Status: Needs review » Reviewed & tested by the community

  • larowlan committed a4f3ff9 on 9.2.x
    Issue #3189463 by quietone, Wim Leers, adityasingh, ayushmishra206: all...
  • larowlan committed 24494c2 on 9.3.x
    Issue #3189463 by quietone, Wim Leers, adityasingh, ayushmishra206: all...
larowlan’s picture

Version: 9.3.x-dev » 9.2.x-dev
Status: Reviewed & tested by the community » Fixed

Committed 24494c2 and pushed to 9.3.x. Thanks!

Backported to 9.2.x

Thanks folks

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

GyD’s picture

FileSize
24.4 KB

We are using a distribution that is still using Drupal 8.9 so I back ported the patch for 8.9