Problem

The "skip_on_value" process plugin allows a comparison against a specific value. However, it does not allow comparing against another source value.

Example 1: Migrate Products

Let's say there is a CSV file that contains 3 columns:

+--------------+-----------+------------+
| name         | sku       | parent_sku |
+--------------+-----------+------------+
| blue sandals | 10001     | 10001      |
| red sandals  | 10002     | 10001      |
+--------------+-----------+------------+

The "blue sandals" product needs to get imported as a Product entity because the "sku" value equals "parent_sku" value. However, the "red sandals" product needs to get imported as a Product Variant entity.

Proposed Resolution

Write a process plugin that supports comparison between two source values on the same row.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

josephdpurcell created an issue. See original summary.

josephdpurcell’s picture

Issue summary: View changes
heddn’s picture

What if you use @other_value, where other_value is another source value?

devarch’s picture

Assuming that in the source block you have defined the fields sku and parent_sku, your request already can be done in the process phase like this:

process:
  _test_if_parent:
    plugin: skip_on_value
    method: row
    equals: false
    value: parent_sku
    source: sku

or if you already have assigned the partent_sku to the destination field:

process:
  dest_parent_sku: parent_sku

  _test_if_parent:
    plugin: skip_on_value
    method: row
    equals: false
    value: "@dest_parent_sku"
    source: sku

I think this issue can be closed.

DuaelFr’s picture

I have a similar issue with taxonomy terms.
In my source, root terms have a "parent" value equal to their "id" value.
In Drupal, root terms has no parent value.

I tried the 3 following methods:

  1.   parent:
        -
          plugin: skip_on_value
          method: process
          value: id
          source: parent
        -
          plugin: migration_lookup
          migration: training_domains
  2.   parent:
        -
          plugin: skip_on_value
          method: process
          value: '@id'
          source: parent
        -
          plugin: migration_lookup
          migration: training_domains
  3.   field_formacode: id
      parent:
        -
          plugin: skip_on_value
          method: process
          value: '@field_formacode'
          source: parent
        -
          plugin: migration_lookup
          migration: training_domains

but neither of these methods work.

While looking at the comments and the code of \Drupal\migrate_plus\Plugin\migrate\process\SkipOnValue, I cannot see how it could work as the source value is directly compared with the $this->configuration['value'] parameter content, without any further transformation or call to the $row object.

DuaelFr’s picture

Here is a patch with a basic improvement of the test coverage.

heddn’s picture

The row object in 8.7 now supports what we want here. See https://www.drupal.org/node/3001535.

DuaelFr’s picture

Hi @heddn!
Thank you for your pointer.

The problem is that the actual plugin takes a static value as parameter instead of a reference to the source/destination. How could we improve my patch to keep the actual behaviour but also allow to use a value from the source or the destination? Should we use a new setting instead of "value"?

heddn’s picture

I'm going back to what I suggested a while ago and trying to recall what I was thinking. I am guessing that I was trying to say that instead of forcing folks to use '@field_foo', if we just used row object's getter, we could make the experience for developers easier by just simply providing the field name, 'field_foo'.

DuaelFr’s picture

Ok, so how can we distinct the current use case, where the "value" is just a value and is never used as a key, and the new use case, that we are asking for, where we want to use a value from the row/destination? Would you create a new setting key for this use case? How would you name it? (I'm asking because I'm not a native eanglish speaker so it is a really difficult task for me ^^)

heddn’s picture

More of what this request was about is coming back to mind. Our options are to either add a flag or config option or create a new plugin (probably extending from this skip_on_value). One of the hard things with extending/creating a new one is naming the thing. But that's what I'd slightly prefer. However, we wouldn't have that problem if we added a config option. So I'm torn. If someone has a good name for a new plugin, let's go that route. Otherwise let's add a config option (and doxygen how to use it) to the existing process plugin.

DuaelFr’s picture

What do you think about "skip_on_row_value" for a new plugin or "row_value" for a new config option?

heddn’s picture

I like either name and would accept either patch. Although I still prefer a new plugin.

benjifisher’s picture

Untested, but something like this should work:

process:
  my_dummy_field:
    -
      plugin: callback
      callable: array_unique
      source:
        - sku
        - parent_sku
    -
      plugin: callback
      callable: count
    -
      plugin: skip_on_value
      method: row
      value: 1
DuaelFr’s picture

Here is the patch where the feature moves to an entirely new process plugin as @heddn prefered.
The interdiff is not attached because it's not relevant.
I hope the testbot is going to like it ;)

heddn’s picture

Version: 8.x-4.x-dev » 8.x-5.x-dev
heddn’s picture

FileSize
12.85 KB

Let's see how this fares on 5.x

danflanagan8’s picture

This is one of the cases currently being addressed very naturally on the following issue: #3259591: Add generic Skip migrate process plugin with various operations

The syntax would be:

skip_when_two_source_values_match:
  plugin: skip_on_condition
  method: row/process
  source: my_source_value
  condition: equals
  configuration:
    property: my_other_source_value

or perhaps this:

skip_when_two_source_values_match:
  plugin: skip_on_condition
  method: row/process
  source: my_source_value
  condition: 
    plugin: equals
    property: my_other_source_value

There's an in_array condition that could be used in place of equals if there are multiple values being considered.

If this looks interesting, please comment here or there. Thanks!

(Note: that issue is currently on core, but it will likely be moved to migrate_plus for consideration)

Matroskeen’s picture

Status: Needs review » Closed (won't fix)
Parent issue: » #3263852: [policy] Conditions plugins in Migrate Plus

I'm closing this one as per #3263852: [policy] Conditions plugins in Migrate Plus. Please check Migrate Conditions project by @danflanagan8.

Thank you!

danflanagan8’s picture

As noted in #19, this feature has been incorporated into the Migrate Conditions module.

An issue has been created in order to credit contributors: #3263890: Credit for Skip when value matches another source value

As an example, here's how you could skip migrating a self-titled album:

process:
  skip_self_titled_album:
    plugin: skip_on_condition
    source: source_album_title
    conditions:
      plugin: equals
      property: source_band_name
    method: row
    message: 'We skip self-titled albums for some reason?'

Please see docs on the equals condition: https://www.drupal.org/docs/contributed-modules/migrate-conditions/migra...

and on the skip_on_condition process plugin: https://www.drupal.org/docs/contributed-modules/migrate-conditions/migra...

Thanks all!

DuaelFr’s picture

That looks great!
Thanks :)

SomebodySysop’s picture

I'm not sure that I understand the resolution here. I wish to have a skip_on option using source fields as the source and value (equal to).

If I understand correctly, the maintainer is saying that the "migrate conditions" module handles this. But, it does not.

process:
  skip_if_group_admin:
    plugin: skip_on_condition
    source: etid
    condition:
      plugin: equals
      property: group_creator
    method: row
    message: 'We skip users who are admins in group.'

This skips all rows when only the first one matched.

Nor does this:

  skip_if_group_admin:
    plugin: skip_on_condition
    method: row
    source: etid
    condition: equals
    configuration:
      property: group_creator

Returns this error: 'Exactly one of value and property must be set when using the equals condition.'

So, how does one 'Skip when value matches another source value'?

danflanagan8’s picture

This is being dealt with in the migrate_conditions queue: #3309134: How to skip row when source value matches another source value