Problem/Motivation

As part of the roadmap of getting migrate tools and upstream drush to play nice together again, we need to provide an easy way to provide shared configuration.

Steps to reproduce

Proposed resolution

This provides a new yaml plugin type that injects itself into the migrate yaml.

Example syntax of new yaml plugin in MODULE_NAME.migrate_shared_configuration.yml:

my_default_configuration:
  source:
    batch_size: 2

Example syntax of an include in a migration.yml:

id: test_stub_migration
label: Test stub migration
include: my_default_configuration
source:

Remaining tasks

User interface changes

API changes

Data model changes

Comments

heddn created an issue. See original summary.

heddn’s picture

Status: Active » Needs review
StatusFileSize
new11.8 KB
heddn’s picture

Status: Needs review » Needs work
Issue tags: +Novice

NW to rename the plugin to "migrate_include/MigrateInclude". Tagging novice, because code refactor like what I just mentioned should be PHP novice stuff.

  1. +++ b/src/MigrateSharedConfigPluginManager.php
    @@ -0,0 +1,65 @@
    + * @code
    
    --- /dev/null
    +++ b/src/SharedConfigIncludeHandler.php
    

    We should also point to src/SharedConfigIncludeHandler.php (which should be renamed to src/MigrateIncludeHandler.php). From here.

  2. +++ b/src/SharedConfigIncludeHandler.php
    @@ -0,0 +1,35 @@
    +final class SharedConfigIncludeHandler {
    

    We should add some documentation here on how this should be invoked in a migrate yaml. Examples of a single and multiple includes would be good.

heddn’s picture

Is migrate_tools the right place for this module? Should it move to plus? Should it be in a new module namespace entirely? I really don't like that last option. It seems troublesome for discoverability of the feature. Plus is maybe the right place. But we're introducing it for the sake of tools, so we can (re)move some of the drush commands into upstream drush.

andregp’s picture

Assigned: Unassigned » andregp

I'll start to work on this issue. I'll begin with the refactoring and the other changes mentioned in comment #3.
Just to be sure @heddn do you want to change even the module name too, right?
I'm novice on PHP and Drupal but will try my best. I may finish it by tonight or tomorrow.

heddn’s picture

I wouldn't do any module naming changes. Those can be done after #3 is addressed.

andregp’s picture

Assigned: andregp » Unassigned
Status: Needs work » Needs review
StatusFileSize
new67.79 KB

Hi @heddn I hope it is useful, this is basically the first issue I'm trying to solve by myself. Please let me know if I made something wrong on the refactoring and how I can improve. All the best!

heddn’s picture

hi @andregp, I think you tried to do a module name change? Let's reverse those parts. And only rename the plugin & classes. Also, when doing incrimental changes like this, an interdiff is really helpful.

andregp’s picture

Assigned: Unassigned » andregp
Status: Needs review » Needs work

Oh sorry, I'll do it then.

andregp’s picture

Assigned: andregp » Unassigned
Status: Needs work » Needs review
StatusFileSize
new18 KB
new21.79 KB
new8.14 KB
new11.17 KB

Thanks for being so patient, so here is the new patch where I changed the MigrateTools class name and refactored the code accordingly. Also an interdiff between this and your patch.

Just in case I also made another patch that includes changes on MigrateToolsTest, MigrateToolsCommands, and MigrateToolsCommandsTest classes names whit a respective interdiff.

Please tell me if I missed something.

heddn’s picture

Status: Needs review » Needs work
+++ b/src/Commands/MigrateToolsCommands.php
@@ -391,7 +391,7 @@ class MigrateToolsCommands extends DrushCommands {
     'update' => FALSE,

This file should not be renamed. Rolling that back would clear up even more noise in the patch.

andregp’s picture

Assigned: Unassigned » andregp

Okay

andregp’s picture

Assigned: andregp » Unassigned
Status: Needs work » Needs review
StatusFileSize
new18.75 KB
new2.85 KB

So I undid the rename on MigrateToolsCommands.php file. Here is the patch and the interdiff

Status: Needs review » Needs work

The last submitted patch, 13: 3250040-13.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

andregp’s picture

Status: Needs work » Needs review
StatusFileSize
new18.2 KB
new3.27 KB

Sorry forgot one place

heddn’s picture

Status: Needs review » Needs work
+++ b/src/Commands/MigrateToolsCommands.php
@@ -15,7 +15,7 @@ use Drupal\migrate\Plugin\RequirementsInterface;
+use Drupal\migrate_tools\MigrateInclude;

This shouldn't rename. It will greatly reduce the noise in this patch too if you back out that change.

andregp’s picture

Are you referring to the change only on that line from MigrateToolsCommands.php? Or do you mean that I also shouldn't rename Drupal\migrate_tools\MigrateTools.php class name to Drupal\migrate_tools\MigrateInclude.php ?

heddn’s picture

Don't rename MigrateTools utility class.

andregp’s picture

Status: Needs work » Needs review
StatusFileSize
new11.87 KB
new2.48 KB
new5.07 KB

I'm sorry, I clearly misunderstood the issue. I thought that renaming the class MigrateTools was part of renaming the module. I apologize.
Here is the new patch where I backed out this change.

heddn’s picture

StatusFileSize
new1.09 KB
new11.88 KB

This is passing green locally. I want to get some feedback on it, but I think it is a distinct possibility this will let us get to a place where migrate tools and drush 10.4+ can co-exist again.

heddn’s picture

Also, refer to the linked change record for more fully details on how this patch would work. And any input on what version 5.2 or 6.0 this feature should be released into.

mradcliffe’s picture

Does migrate_tools need to add its own schema file in order to modify the migrate_plus config_entity schema to account for the new key?

Something like the following, right?

migrate_plus.migration.*.third_party.migrate_tools:
  type: mapping
  label: 'Shared configuration'
  mapping:
    include:
      type: mapping|sequence?
      label: 'Included configuration?'
heddn’s picture

re #22:

Migrate_tools could add a schema entry. But it isn't strictly required. You can use shared config with migrate plus config entities or with core's yml based migrate plugins. I'm using less and less of migrate plus config entities these days, so I hadn't really thought about needing a schema entry. Does it need one?

heddn’s picture

Issue tags: -Novice
StatusFileSize
new563 bytes
new12.43 KB

I'm leaning towards put this into a 6.x branch so I don't have to worry about breaking as many things. Ergo, bumping a few things in the composer.json/.info.yml.

Status: Needs review » Needs work

The last submitted patch, 24: 3250040-24.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

heddn’s picture

Status: Needs work » Needs review
StatusFileSize
new16.2 KB
new3.77 KB

Status: Needs review » Needs work

The last submitted patch, 26: 3250040-26.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

heddn’s picture

Version: 8.x-5.x-dev » 6.0.x-dev
Status: Needs work » Fixed

Automated testing will fail until Drush 11.0.6 is released. But those failures are related to issues in Drush's testing classes, not the classes under test. Tests pass as-is with Drush ^10.6 on local. Watch for a quick-follow to make the code work with PHP 8.1.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.