In the Get migrate processor plug-in Get->multiple() returns null when source is array, and as array, there will be multiple values, so it should return TRUE.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

edurenye created an issue. See original summary.

edurenye’s picture

Assigned: Unassigned » edurenye
Status: Active » Needs review
FileSize
519 bytes

Done.

Status: Needs review » Needs work

The last submitted patch, 2: get_migration_plugin-2884019-2.patch, failed testing. View results

edurenye’s picture

I found this issue trying to find why the processors with handle_multiple false, with multiple sources where receiving an array, when processRow is suposed to handle that, and it was because of the Get processor setting multiple from FALSE to NULL in MigrateExecutable->processRow()

mikeryan’s picture

Status: Needs work » Postponed (maintainer needs more info)

Could you elaborate some more on the problem you're seeing? Perhaps an example .yml that isn't working as you expect?

edurenye’s picture

Status: Postponed (maintainer needs more info) » Needs work

I was importing a product entity and I had this processor to fill a field with products of the same kind but different color:

field_also_available:
    plugin: migration_lookup
    migration: migrate_cases_csv
    source:
      - color_available_1
      - color_available_2

The current migration is migrate_cases_csv.

As migration_lookup has handle_multiple to FALSE (default) and field_also_available is a multivalued field, I expect the processor to iterate the sources in MigrateExecutable->processRow() and fill a new entry for each source, but as in reality this executes first the Get plugin and then the migration_lookup plugin, In this fragment of code inside processRow():

      $multiple = FALSE;
      /** @var $plugin \Drupal\migrate\Plugin\MigrateProcessInterface */
      foreach ($plugins as $plugin) {
        $definition = $plugin->getPluginDefinition();
        // Many plugins expect a scalar value but the current value of the
        // pipeline might be multiple scalars (this is set by the previous
        // plugin) and in this case the current value needs to be iterated
        // and each scalar separately transformed.
        if ($multiple && !$definition['handle_multiples']) {

Get in the first iteration is setting multiple to NULL when it is returning an array, so should set it to TRUE.
But setting it to null acts like with FALSE, so it gives the value to migrate_lookup without iterate over it, and migrate_lookup behaves unexpectedly as it just returns the first that it finds.

edurenye’s picture

The failing tests are because there are other process plugins that should have handle_multiples TRUE, as the can do it, but they where relying in this wrong behavior of Get.
One example that I'm fixing here is FileCopy, that handles multiples as expects an array of source and destination, but did not have handle_multiples.

Status: Needs review » Needs work

The last submitted patch, 7: get_migration_plugin-2884019-7.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

mikeryan’s picture

Category: Bug report » Support request
Status: Needs work » Postponed (maintainer needs more info)

So, here:

    source:
      - color_available_1
      - color_available_2

You are expecting that the two fields will be merged into a single array of source values. That's not how it works - rather, migration_lookup takes them as two parts of a multi-value key to look up (i.e., in the migration map table it'll want to look for sourceid1=color_available_1 and sourceid2=color_available_2). You need to separately merge those values into a single value array to pass to migration_lookup. I believe this will work (assuming those are scalar values):

field_also_available:
    -
      plugin: get
      source:
        - color_available_1
        - color_available_2
    -
      plugin: migration_lookup
      migration: migrate_cases_csv
edurenye’s picture

This is not working.
As multiple by default is FALSE even with array, because of that, Get gets called just once, but as Get does not care of value but uses the config property source, it gets the array and returns it, so there is no actual difference between
this:

field_also_available:
    -
      plugin: get
      source:
        - color_available_1
        - color_available_2
    -
      plugin: migration_lookup
      migration: migrate_cases_csv

and this:

field_also_available:
    plugin: migration_lookup
    migration: migrate_cases_csv
    source:
      - color_available_1
      - color_available_2

I still think this is a bug in Drupal as the behavior of handle_multiple does not match the documentation https://api.drupal.org/api/drupal/core%21modules%21migrate%21src%21Annotation%21MigrateProcessPlugin.php/property/MigrateProcessPlugin%3A%3Ahandle_multiples/8.3.x

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

heddn’s picture

Status: Postponed (maintainer needs more info) » Closed (cannot reproduce)

If we can reproduce this, please re-open.

edurenye’s picture

Category: Support request » Bug report
Status: Closed (cannot reproduce) » Active

I still think that this is a bug as the MigrateProcessPlugin behaves different from the documentation, so we have to change documentation or the plug-in.
And as I said the response from 9 is not working.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

quietone’s picture

Status: Active » Closed (cannot reproduce)
Issue tags: +Bug Smash Initiative

I read the issue and I also don't see the bug here or a problem with the documentation. If field_also_available is a multi-value field then I would expect to the sub_process plugin to be used to get the values for the other colors.

Steps to reproduce 5 years ago and that has not been provided nor has suggestions for changes to documentation. I'm marking the issue "Closed (cannot reproduce)". If anyone can provide complete steps to reproduce the issue (starting from "Install Drupal core"), document those steps in the issue summary and set the issue status back to "Active".

Thanks.