If I mess up my migration's process, then the not very helpful error is something along the lines of:

>Invalid argument supplied for foreach() Migration.php:620 [warning]

This doesn't say:

- which migration is at fault (if the migration being run uses lookups and stubs, then it could be one of the lookup migrations)
- which process item is at fault

This could be fixed by getProcessNormalized doing a sanity check:

      if (!is_array($configuration)) {
        $migration_id = $this->getPluginId();
        throw new \Exception("Invalid process for destination $destination in migration $migration_id.");
      }

Manual test results
I added 'test0: 0' to the process pipeline iin the system_logging migration. then ran `drush im syste_logging`.

$ drush mim system_logging
 [warning] Invalid argument supplied for foreach() Migration.php:622
 [warning] Invalid argument supplied for foreach() Migration.php:622
 [warning] Invalid argument supplied for foreach() Migration.php:622
 [warning] Invalid argument supplied for foreach() Migration.php:622
 [notice] Processed 1 item (0 created, 0 updated, 1 failed, 0 ignored) - done with 'system_logging'

I applied the patch from #16 and now get a useful message.

$ drush ms system_logging                                        
In Migration.php line 391:
                                                                         
  Invalid process for destination 'test0' in migration 'system_logging'  
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

joachim created an issue. See original summary.

quietone’s picture

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Wim Leers’s picture

quietone’s picture

Yes, this would be a nice addition and catch errors early. I have added the code in the IS to Migration.php and added a unit test.

Status: Needs review » Needs work

The last submitted patch, 7: 2969231-7.patch, failed testing. View results

quietone’s picture

Status: Needs work » Needs review
FileSize
2.86 KB
3.61 KB

So, there is an existing Kernel test for this, let's just build on that instead of adding a new test.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

NickDickinsonWilde’s picture

Version: 9.1.x-dev » 8.9.x-dev
Status: Needs review » Reviewed & tested by the community

Looks good to me - and works beautifully.
I've marked as RTBC, however, I think there is a chance, that

        if (!is_array($configurations) && !$this->processPlugins[$index][$property]) {
          throw new MigrateException(sprintf("Process configuration for '$property' must be an array", $property));
        }

from /core/modules/migrate/src/Plugin/Migration.php (line 367 or so) should be removed as unnecessary due to getProcessNormalized() running first. Unless I'm missing some instance that would only run the latter?

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

quietone’s picture

Version: 9.1.x-dev » 8.9.x-dev
Status: Reviewed & tested by the community » Needs review
FileSize
968 bytes
3.57 KB

thanks, I meant to come back and mention that and clearly I forgot. I see no reason to not remove that code in this patch.

NickDickinsonWilde’s picture

Status: Needs review » Reviewed & tested by the community

LGTM

xjm’s picture

Priority: Minor » Normal
Status: Reviewed & tested by the community » Needs review
Issue tags: +Needs followup

Regarding #11 / #13.

+++ b/core/modules/migrate/src/Plugin/Migration.php
@@ -364,9 +364,6 @@ public function getProcessPlugins(array $process = NULL) {
       foreach ($this->getProcessNormalized($process) as $property => $configurations) {
         $this->processPlugins[$index][$property] = [];
-        if (!is_array($configurations) && !$this->processPlugins[$index][$property]) {
-          throw new MigrateException(sprintf("Process configuration for '$property' must be an array", $property));
-        }
         foreach ($configurations as $configuration) {

So, before, we were throwing an exception if $configurations wasn't an array right before foreaching it.
getProcessNormalized() does this:

protected function getProcessNormalized(array $process) {
  $normalized_configurations = [];
  foreach ($process as $destination => $configuration) {
    if (is_string($configuration)) {
      $configuration = [
        'plugin' => 'get',
        'source' => $configuration,
      ];
    }
    if (isset($configuration['plugin'])) {
      $configuration = [
        $configuration,
      ];
    }
    $normalized_configurations[$destination] = $configuration;
  }
  return $normalized_configurations;
}

If any configuration child key of $process is a string, it converts it to an array. But what if it's a different kind of scalar, or an object? Under those circumstances, getProcessNormalized would not be guarantee that the child keys of what it returns are also arrays.

It's slightly confusing because in getProcessNormalized() the return array is called "configurations" (plural) with each child being a configuration (singular). But then in the caller, those individual children are each called "configurations" (plural).

This all might be fine, but I'd suggest leaving the earlier exception in for now and addressing a proposal to remove that hunk in a followup.

quietone’s picture

RE-uploading the patch from 9 which does not remove the check from getProcessPlugins().

Follow up made, #3133139: Remove is_array check in getProcessPlugins.

NickDickinsonWilde’s picture

Status: Needs review » Reviewed & tested by the community

re-rtbc-ing.

quietone’s picture

Issue summary: View changes

Add manual testing to the IS

alexpott’s picture

Status: Reviewed & tested by the community » Fixed

Committed and pushed 93545bd537 to 9.1.x and 145d174fdb to 9.0.x and b1007e34e4 to 8.9.x. Thanks!

This is nice improvement that helps people work out what's going wrong - always a good thing with migrations!

diff --git a/core/modules/migrate/tests/src/Kernel/Plugin/MigrationTest.php b/core/modules/migrate/tests/src/Kernel/Plugin/MigrationTest.php
index 5b730be230..c58df28af5 100644
--- a/core/modules/migrate/tests/src/Kernel/Plugin/MigrationTest.php
+++ b/core/modules/migrate/tests/src/Kernel/Plugin/MigrationTest.php
@@ -42,7 +42,7 @@ public function testGetProcessPluginsException() {
   }
 
   /**
-   * Tests Migration::getDestinationPlugin()
+   * Tests Migration::getProcessPlugins()
    *
    * @param array $process
    *   The migration process pipeline.

Fixed the test description on commit.

  • alexpott committed 93545bd on 9.1.x
    Issue #2969231 by quietone, NickDickinsonWilde, joachim, xjm: errors in...

  • alexpott committed 145d174 on 9.0.x
    Issue #2969231 by quietone, NickDickinsonWilde, joachim, xjm: errors in...

  • alexpott committed b1007e3 on 8.9.x
    Issue #2969231 by quietone, NickDickinsonWilde, joachim, xjm: errors in...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.