Problem/Motivation
The core MigrateStub instance allows stubs to create stub, however the MigMag version doesn't.
Proposed resolution
Introduce a new no_stub_limit configuration for the migmag_lookup plugin, with something along the lines of this:
process:
target_id:
plugin: migmag_lookup
migration: asset_migration
source: asset_id
# Even if this migration entry is for a stub, allow it to create more stubs during its lookup.
no_stub_limit: true
Remaining tasks
Provide MR
- Add test cases
API changes
N/A. Existing behaviour should stay the same.
It's an opt-in behaviour to maintain backwards compatibility.
Comments
Comment #2
huzookaI explicitly designed the stub mechanism like this; because this way the only option to run into and endless loop if the entity references itself.
Do you have a concept or idea how to prevent circular stubbing when somebody uses the feature you proposed?
Comment #4
codebymikey commentedYeah, I understand. I've ran into circular stubbing issues myself (when I was still using core's MigrateStub).
And I think it's more of an issue with the migration itself, and it should have logic in place (e.g. through the migrate_conditions module) to stop it from stubbing itself e.g. Ensure that you do a
no_stublook up initially so if it's already been stubbed, that ID is used, rather than attempting to stub itself.But yeah, I understand the concern either way (and everyone's migration is different, mine however requires entity stubs to be able to create stubs just so the second migration creates the entity I want to reference).
We'd keep a list of currently currently stubbed entities using a key like:
"$migration_id:$source_id", and use that avoid potential duplicates if it finds that the same entity is already being stubbed.The default behaviour is to limit it to at most 1 entity stub at a time (current behaviour), then configurations which require 'stub entity -> stub entity' behaviour can do so by updating their
settings.phpto either0(no limit), or increase the limit to10or whatever maximum they forsee themselves using if they're worried about circular stubs.I've created a Draft MR with my implementation - still needs test, but we probably need to fix the existing failing tests first, so we can confirm that it doesn't break anything.
edit: On further thoughts, think its better to be set for each individual
migmag_lookupprocess, rather than a global setting (as I wanted that stub -> stub behaviour on a specific process).Comment #5
codebymikey commentedThe MR also includes the edge case Exception handling logic raised in #3571993: MigMagMigrateStub::$mainStubProcessHasStarted can be stuck as TRUE if the stub throws an exception downstream
Comment #6
codebymikey commentedUpon further reflection and understanding the code a bit more, I updated the MR with a per-process configuration instead.
It's more scalable as the behaviour I wanted was for a specific lookup, rather than globally.