Problem/Motivation
When using the migration_lookup process plugin and providing multiple migrations to search in, the plugin may exit early due to NULL destination ids from skipped rows being returned by the SqlMap plugin.
Example setup:
- A migration lookup using multiple migrations
- plugin: migration_lookup migration: - node_by_x - node_by_y - node_by_z no_stub: true source: nid - Source ids are valid for multiple of these migrations
- Each migration is only responsible for a subset of its source rows, so these migrations have some ignored rows in the map
_valid: - plugin: static_map map: 12: true 123: true 1234: true default_value: '' source: uid - plugin: skip_on_empty method: row
This code in MigrationLookup::transform is perfectly fine:
// Break out of the loop as soon as a destination ID is found.
if ($destination_ids = $migration
->getIdMap()
->lookupDestinationId($source_id_values[$migration_id])) {
break;
}
However, in a setup where a migration may skip rows, the Sql::lookupDestinationIds method will treat skipped rows as if they were imported, but with NULL destination id(s), breaking any further lookups for migrations that may have actually imported the row.
Proposed resolution
Only return rows from the migrate map that were handled by the migration (= are not ignored) in Sql::lookupDestinationIds.
As a workaround, we've been chaining lookups for each migration individually, using a custom processor to check for empty destination ids.
Remaining tasks
Review, add tests.
Optionally decide whether other map row statuses should be excluded as well (e.g. failed rows).
| Comment | File | Size | Author |
|---|---|---|---|
| #2 | drupal-migration_lookup_ignored-2954192-2.patch | 852 bytes | ckaotik |
Comments
Comment #2
ckaotikComment #7
jcnventuraEven if this issue preceded it, closing as a duplicate of #3095270: Sql Migration map returns nonsensical destination ids when looking up against a source id row that failed or was ignored.