Problem/Motivation

In #2876085: Before upgrading, audit for potential ID conflicts we needed to have access to the MigrationPluginManager inside Drupal\migrate\Plugin\migrate\id_map\Sql. To avoid a BC break, we used a singleton behind a protected accessor with a todo to resolve in 9.0.

Proposed resolution

Add the MigrationPluginManager by dependency injection.

Remaining tasks

Do it.

User interface changes

None.

API changes

The Sql class would have a new parameter to its constructor, the MigrationPluginManager.

Data model changes

None.

CommentFileSizeAuthor
#39 interdiff-37-39.txt693 bytesdanflanagan8
#39 10.0.x-2919158-39.patch8.93 KBdanflanagan8
#39 2919158-39.patch12.22 KBdanflanagan8
#37 interdiff-10x-33-37.txt1.13 KBdanflanagan8
#37 10.0.x-2919158-37.patch8.93 KBdanflanagan8
#37 interdiff-33-37.txt855 bytesdanflanagan8
#37 2919158-37.patch12.17 KBdanflanagan8
#33 interdiff-33-9.5.x-10.0.x.txt4.96 KBdanflanagan8
#33 interdiff-26-33.txt1.24 KBdanflanagan8
#33 10.0.x-2919158-33.patch9.49 KBdanflanagan8
#33 2919158-33.patch12.15 KBdanflanagan8
#26 interdiff_24-26.txt1.15 KBdanflanagan8
#26 2919158-26.patch10.91 KBdanflanagan8
#24 interdiff_22-24.txt1.78 KBdanflanagan8
#24 2919158-24.patch10.91 KBdanflanagan8
#22 interdiff_19-22.txt2.11 KBdanflanagan8
#22 2919158-22.patch10.91 KBdanflanagan8
#19 2919158-19.patch10.89 KBquietone
#19 interdiff-16-19.txt2.27 KBquietone
#16 2919158-16.patch9.39 KBquietone
#16 interdiff-12-16.txt4.02 KBquietone
#15 2919158-15.patch9.38 KBquietone
#15 interdiff-12-15.txt3.79 KBquietone
#12 interdiff_10-12.txt3.27 KBdanflanagan8
#12 2919158-12.patch9.24 KBdanflanagan8
#10 2919158-10.patch6.8 KBdanflanagan8
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

maxocub created an issue. See original summary.

maxocub’s picture

Title: Add the MigrationPluginManager to Sql » Add the MigrationPluginManager to Drupal\migrate\Plugin\migrate\id_map\Sql
heddn’s picture

Status: Postponed » Active

I don't think this is blocked any more.

maxocub’s picture

Status: Active » Postponed

This is a BC break so it will only be possible in 9.x.

Version: 9.x-dev » 9.0.x-dev

The 9.0.x branch will open for development soon, and the placeholder 9.x branch should no longer be used. Only issues that require a new major version should be filed against 9.0.x (for example, removing deprecated code or updating dependency major versions). New developments and disruptive changes that are allowed in a minor version should be filed against 8.9.x, and significant new features will be moved to 9.1.x at committer discretion. For more information see the Allowed changes during the Drupal 8 and 9 release cycles and the Drupal 9.0.0 release plan.

xjm’s picture

Version: 9.0.x-dev » 9.1.x-dev
Status: Postponed » Active

Since 8.9.x and 9.0.x are now in beta, I'm moving this to 9.1.x. Is there a way to make this change with BC/deprecation for the continuous upgrade path? Thanks!

The issue probably also does not need to be postponed since it doesn't appear to be postponed on anything specific. Marking active.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

danflanagan8’s picture

Status: Active » Needs review
FileSize
6.8 KB

Here's a shot at taking care of this. This has an update to an existing Unit test so that it doesn't fail due to the missing service plugin.manager.migrate service. In normal situations there's not a BC problem. It's just the nature of unit tests.

I ran MigrateSqlIdMapTest locally and it's passing. If the deprecation is as graceful as I think it is I wouldn't expect anything else to fail. But we'll see!

Status: Needs review » Needs work

The last submitted patch, 10: 2919158-10.patch, failed testing. View results

danflanagan8’s picture

Status: Needs work » Needs review
FileSize
9.24 KB
3.27 KB

Not too graceful apparently! This should be better.

mikelutz’s picture

Status: Needs review » Needs work
Issue tags: -Migrate BC break +Needs change record
+++ b/core/modules/migrate/src/Plugin/migrate/id_map/Sql.php
@@ -993,18 +1010,6 @@ public function valid() {
-  protected function getMigrationPluginManager() {
-    return \Drupal::service('plugin.manager.migration');
-  }

Need to deprecate this for removal as well, rather than just deleting it.

mikelutz’s picture

Issue tags: +Portland2022
quietone’s picture

Status: Needs work » Needs review
Issue tags: +Migrate critical
FileSize
3.79 KB
9.38 KB

Hoping to nudge this along. Adding tag because this has deprecations for 9.4.

  • 13. Started a CR so we have the URL for the deprecation messages.
  • 13. Done, Added the deprecation message to getMigrationPluginManager().
  • And changed the name of $migrate_manager and $migrateManager to include the word plugin so that it is very clear that this manager is not the MigratePluginManager.
quietone’s picture

Had a coding standards errors, just ignore the patch in #15

mikelutz’s picture

Still need to trigger a deprecation error in getMigrationPluginManager, I think, unless we are doing that automatically from the annotation now. I confess it's been a while since I've done a deprecation..

mikelutz’s picture

Status: Needs review » Needs work
quietone’s picture

Status: Needs work » Needs review
FileSize
2.27 KB
10.89 KB

Writing the test for that took most of the time :-)

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

mikelutz’s picture

Status: Needs review » Needs work
+++ b/core/modules/migrate/src/Plugin/migrate/id_map/Sql.php
@@ -996,12 +1013,16 @@ public function valid() {
+    @trigger_error("deprecated in drupal:9.4.0 and is removed from drupal:10.0.0. No direct replacement is provided. See https://www.drupal.org/node/3277306", E_USER_DEPRECATED);

Let's push to get this in 9.5. We try to avoid deprecations in X.last, but this class is not one that would normally be extended, so it should be fine.

Also, there is a replacement, `use $this-migrationPluginManager instead`

danflanagan8’s picture

Status: Needs work » Needs review
FileSize
10.91 KB
2.11 KB

Here are the updates requested by @mikelutz.

Status: Needs review » Needs work

The last submitted patch, 22: 2919158-22.patch, failed testing. View results

danflanagan8’s picture

Status: Needs work » Needs review
FileSize
10.91 KB
1.78 KB

Ugh. This patch changes the double quotes to single quotes so there's no interpolation in the deprecation warning.

Spokje’s picture

Status: Needs review » Needs work

*cough*

--- a/core/modules/migrate/src/Plugin/migrate/id_map/Sql.php
+++ b/core/modules/migrate/src/Plugin/migrate/id_map/Sql.php
@@ -176,6 +186,12 @@ public function __construct(array $configuration, $plugin_id, $plugin_definition
     $this->mapTableName = mb_substr($this->mapTableName, 0, 63 - $prefix_length);
     $this->messageTableName = 'migrate_message_' . mb_strtolower($machine_name);
     $this->messageTableName = mb_substr($this->messageTableName, 0, 63 - $prefix_length);
+
+    if (!$migration_plugin_manager) {
+      @trigger_error('Calling Sql::__construct() without the $migration_manager argument is deprecated in drupal:9.4.0 and the $migration_manager argument will be required in drupal:10.0.0. See https://www.drupal.org/node/3277306', E_USER_DEPRECATED);

Still mentioning is deprecated in drupal:9.4.0
*cough*

danflanagan8’s picture

Status: Needs work » Needs review
FileSize
10.91 KB
1.15 KB

Oof. Thanks, @Spokje. Let's try this one.

mikelutz’s picture

Status: Needs review » Needs work
+++ b/core/modules/migrate/src/Plugin/migrate/id_map/Sql.php
@@ -176,6 +186,12 @@ public function __construct(array $configuration, $plugin_id, $plugin_definition
+      @trigger_error('Calling Sql::__construct() without the $migration_manager argument is deprecated in drupal:9.5.0 and the $migration_manager argument will be required in drupal:10.0.0. See https://www.drupal.org/node/3277306', E_USER_DEPRECATED);

We still need a test to trigger this deprecation

danflanagan8’s picture

Status: Needs work » Needs review

@mikelutz, I don't have a good reference for this claim, but deprecations related to new arguments in the constructor are not generally covered by automated tests. There are a handful of tests for stuff like that, but mostly not. I think we should be ok.

mikelutz’s picture

Hmm, I haven't heard of that policy change. I've always written a test when I've added parameters to a constructor.

I don't see an exception listed in the how to deprecate instructions, which are right below the list of what can be deprecated, which includes constructor parameter additions. I don't see anything either way in the specifc constructor parameter addition instructions, which isn't unexpected, none of the other sections really mention testing either. Is this the most recent deprecation policy, or is there another policy that I'm missing here?

Spokje’s picture

Unsure about any policy change on deprecations, but at the very least I think this issue needs a 10.0.x patch, with removal of the deprecations made in the 9.5.x patch.

mikelutz’s picture

Status: Needs review » Needs work

Agreed with @Spokje, though there is a chance the RMs might want to put this in the new 10.1.x anyway, so we should probably find out.

Spokje’s picture

Sense = made.

danflanagan8’s picture

Here's a new patch for 9.5.x that includes a test for the deprecation in the constructor. I'm also including a patch for 10.0.x with the deprecations removed.

danflanagan8’s picture

@mikelutzt, @Spokje, any ideas on the custom commands failure in the 10.0.x patch in #33?

--- Commands Executed ---
core/scripts/dev/commit-code-check.sh --drupalci
Return Code: 1
--- Output ---
Drupal's JavaScript development dependencies are not installed or cannot be resolved. Run 'yarn install' inside the core directory, or 'yarn check -s' to list other errors.
--- Errors ---
Spokje’s picture

I've seen it before, but never found out what's exactly wrong. I always blame it on the particular GitLab runner doing the test being wonky and retry, hoping it gets run on another one.

In the other retry there are real dragons:

10.0.x-2919158-33.patch test with PHP 8.1 & MySQL 5.7

Created 29 Jun 2022 at 17:34 CEST. danflanagan8 added custom test. Updated 29 Jun 2022 at 17:42 CEST.

Custom Commands Failed - View results on dispatcher

----------------------------------------------------------------------------------------------------

Running PHPStan on *all* files.
 ------ ----------------------------------------------------------------------- 
  Line   core/modules/migrate/tests/src/Unit/TestSqlIdMap.php                   
 ------ ----------------------------------------------------------------------- 
  98     Call to an undefined static method                                     
         Drupal\migrate\Plugin\migrate\id_map\Sql::getMigrationPluginManager()  
         .                                                                      
 ------ ----------------------------------------------------------------------- 


 [ERROR] Found 1 error                                                          


PHPStan: failed


FILE: ...html/core/modules/migrate/tests/src/Unit/MigrateSqlIdMapTest.php
----------------------------------------------------------------------
FOUND 0 ERRORS AND 2 WARNINGS AFFECTING 2 LINES
----------------------------------------------------------------------
 5 | WARNING | [x] Unused use statement
 7 | WARNING | [x] Unused use statement
----------------------------------------------------------------------
PHPCBF CAN FIX THE 2 MARKED SNIFF VIOLATIONS AUTOMATICALLY
----------------------------------------------------------------------

The last submitted patch, 33: 2919158-33.patch, failed testing. View results

danflanagan8’s picture

That was all kinds of bad. This should be better. Apologies.

@mikelutz had too much faith in my abilities when he said in Slack:

If you took the time you’ve spent arguing and just written the test, you’d be done by now.

I was right to try to get around doing this!

mikelutz’s picture

Status: Needs review » Needs work

Looking good, one nit:

+++ b/core/modules/migrate/src/Plugin/migrate/id_map/Sql.php
@@ -996,12 +1013,16 @@ public function valid() {
   protected function getMigrationPluginManager() {
+    @trigger_error('deprecated in drupal:9.5.0 and is removed from drupal:10.0.0. Use $this->migrationPluginManager instead. See https://www.drupal.org/node/3277306', E_USER_DEPRECATED);
     return \Drupal::service('plugin.manager.migration');
   }
 

This should return $this-migrationPluginManager. We've ensured it's initialized in the constructor to the service in the container, so no need to go back to the container again here.

danflanagan8’s picture

I think that's a worthy nit, @mikelutz. Here's the updated patch for 9.5.x. I'm also re-posting the 10.0.x patch from #37 but with a new name, just to try to keep the most up-to-date patches in the same place.

mikelutz’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: -Needs change record updates

Thanks. I tweaked the CR, lets get this in front of committers and hope they will allow a small deprecation like this in 9.5.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 39: 10.0.x-2919158-39.patch, failed testing. View results

danflanagan8’s picture

Status: Needs work » Reviewed & tested by the community

That recent fail was clearly unrelated to this issue:

1) Drupal\Tests\quickedit\FunctionalJavascript\InlineFormErrorsIntegrationTest::testDisabledInlineFormErrors
WebDriver\Exception\JavaScriptError: javascript error: Drupal.quickedit.editors[editorName] is not a constructor
  (Session info: headless chrome=98.0.4758.102)
  (Driver info: chromedriver=98.0.4758.102 (273bf7ac8c909cde36982d27f66f3c70846a3718-refs/branch-heads/4758@{#1151}),platform=Linux 4.9.0-0.bpo.6-amd64 x86_64)

Resetting status to RTBC.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 39: 10.0.x-2919158-39.patch, failed testing. View results

danflanagan8’s picture

Status: Needs work » Reviewed & tested by the community

Unrelated failure in Drupal\Tests\book\Functional\BookTest::testGetTableOfContents. Resetting to RTBC.

alexpott’s picture

Status: Reviewed & tested by the community » Fixed

Discussed with @catch as we agreed that deprecating in Drupal 9.5 but marking for removal for D11 is the way to go here.

diff --git a/core/modules/migrate/src/Plugin/migrate/id_map/Sql.php b/core/modules/migrate/src/Plugin/migrate/id_map/Sql.php
index e648a8e586..4b76e4fbc6 100644
--- a/core/modules/migrate/src/Plugin/migrate/id_map/Sql.php
+++ b/core/modules/migrate/src/Plugin/migrate/id_map/Sql.php
@@ -188,7 +188,7 @@ public function __construct(array $configuration, $plugin_id, $plugin_definition
     $this->messageTableName = mb_substr($this->messageTableName, 0, 63 - $prefix_length);
 
     if (!$migration_plugin_manager) {
-      @trigger_error('Calling Sql::__construct() without the $migration_manager argument is deprecated in drupal:9.5.0 and the $migration_manager argument will be required in drupal:10.0.0. See https://www.drupal.org/node/3277306', E_USER_DEPRECATED);
+      @trigger_error('Calling Sql::__construct() without the $migration_manager argument is deprecated in drupal:9.5.0 and the $migration_manager argument will be required in drupal:11.0.0. See https://www.drupal.org/node/3277306', E_USER_DEPRECATED);
       $migration_plugin_manager = \Drupal::service('plugin.manager.migration');
     }
     $this->migrationPluginManager = $migration_plugin_manager;
@@ -1017,13 +1017,13 @@ public function valid() {
    * @return \Drupal\migrate\Plugin\MigrationPluginManagerInterface
    *   The migration plugin manager.
    *
-   * @deprecated in drupal:9.5.0 and is removed from drupal:10.0.0. Use
-   * $this->migrationPluginManager instead.
+   * @deprecated in drupal:9.5.0 and is removed from drupal:11.0.0. Use
+   *   $this->migrationPluginManager instead.
    *
    * @see https://www.drupal.org/node/3277306
    */
   protected function getMigrationPluginManager() {
-    @trigger_error('deprecated in drupal:9.5.0 and is removed from drupal:10.0.0. Use $this->migrationPluginManager instead. See https://www.drupal.org/node/3277306', E_USER_DEPRECATED);
+    @trigger_error('deprecated in drupal:9.5.0 and is removed from drupal:11.0.0. Use $this->migrationPluginManager instead. See https://www.drupal.org/node/3277306', E_USER_DEPRECATED);
     return $this->migrationPluginManager;
   }
 
diff --git a/core/modules/migrate/tests/src/Kernel/Plugin/id_map/SqlDeprecationTest.php b/core/modules/migrate/tests/src/Kernel/Plugin/id_map/SqlDeprecationTest.php
index 369252deff..4bdab83ec4 100644
--- a/core/modules/migrate/tests/src/Kernel/Plugin/id_map/SqlDeprecationTest.php
+++ b/core/modules/migrate/tests/src/Kernel/Plugin/id_map/SqlDeprecationTest.php
@@ -23,7 +23,7 @@ class SqlDeprecationTest extends KernelTestBase {
    */
   public function testOptionalParametersDeprecation(): void {
     $migration = $this->prophesize('\Drupal\migrate\Plugin\MigrationInterface')->reveal();
-    $this->expectDeprecation('Calling Sql::__construct() without the $migration_manager argument is deprecated in drupal:9.5.0 and the $migration_manager argument will be required in drupal:10.0.0. See https://www.drupal.org/node/3277306');
+    $this->expectDeprecation('Calling Sql::__construct() without the $migration_manager argument is deprecated in drupal:9.5.0 and the $migration_manager argument will be required in drupal:11.0.0. See https://www.drupal.org/node/3277306');
     new Sql(
       [],
       'sql',
diff --git a/core/modules/migrate/tests/src/Unit/MigrateSqlIdMapTest.php b/core/modules/migrate/tests/src/Unit/MigrateSqlIdMapTest.php
index 83669708d0..f05ed19c78 100644
--- a/core/modules/migrate/tests/src/Unit/MigrateSqlIdMapTest.php
+++ b/core/modules/migrate/tests/src/Unit/MigrateSqlIdMapTest.php
@@ -1190,7 +1190,7 @@ public function testGetMigrationPluginManagerDeprecation() {
     $container->set('plugin.manager.migration', $migration_plugin_manager);
     \Drupal::setContainer($container);
 
-    $this->expectDeprecation('deprecated in drupal:9.5.0 and is removed from drupal:10.0.0. Use $this->migrationPluginManager instead. See https://www.drupal.org/node/3277306');
+    $this->expectDeprecation('deprecated in drupal:9.5.0 and is removed from drupal:11.0.0. Use $this->migrationPluginManager instead. See https://www.drupal.org/node/3277306');
     $id_map = $this->getIdMap();
     $id_map->getMigrationPluginManager();
   }

Fixed this on commit :)

diff --git a/core/modules/migrate/tests/src/Unit/TestSqlIdMap.php b/core/modules/migrate/tests/src/Unit/TestSqlIdMap.php
index be99becdcc..433b3907ee 100644
--- a/core/modules/migrate/tests/src/Unit/TestSqlIdMap.php
+++ b/core/modules/migrate/tests/src/Unit/TestSqlIdMap.php
@@ -95,7 +95,7 @@ public function ensureTables() {
    * {@inheritdoc}
    */
   public function getMigrationPluginManager() {
-    parent::getMigrationPluginManager();
+    return parent::getMigrationPluginManager();
   }
 
 }

Fixed this PHPStan error on commit. Nice catch PHPStan!

Committed and pushed 275a32813b to 10.1.x and abea4c7b5d to 10.0.x and 409a884bd7 to 9.5.x. Thanks!

  • alexpott committed 275a328 on 10.1.x
    Issue #2919158 by danflanagan8, quietone, mikelutz, maxocub, Spokje: Add...

  • alexpott committed abea4c7 on 10.0.x
    Issue #2919158 by danflanagan8, quietone, mikelutz, maxocub, Spokje: Add...

  • alexpott committed 409a884 on 9.5.x
    Issue #2919158 by danflanagan8, quietone, mikelutz, maxocub, Spokje: Add...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.