This issue is spun off from #2908282: Throw exception for source plugins without a source_module property.
Problem/Motivation
There are several implicitily required properties to destination, source and process plugins. Rather than waiting for someone to WSOD when they miss one of these implicit requirements, we should throw a loud exception.
The MigrateDestination annotation defines a destination_module property which needs to identify the system which will contain the processed, migrated data. In Migrate Drupal's case, this property is used to identify the D8 module which will "own" the migrated data.
When providing a strong API, it's important to tell consumers when they are doing something wrong. For Migrate Drupal's purposes (our main use case), this property is required in order to inform users whether their D6/D7 data will be migrated into Drupal 8. Therefore, this blocks the completion of the core migration path to Drupal 8, and is Migrate-critical.
Proposed resolution
- Port the destination-specific code from #2908282-53: Throw exception for source plugins without a source_module property into this issue
- Iterate on it until it's ready to commit. Consider inputs from:
- #2908282-64: Throw exception for source plugins without a source_module property,
- #2908282-65: Throw exception for source plugins without a source_module property &
- #2908282-66: Throw exception for source plugins without a source_module property
- Commit it.
Remaining tasks
That thing I just said.
User interface changes
None.
API changes
The destination_module property will be required on all Migrate destination plugins. So this constitutes a BC break.
Data model changes
None.
Comment | File | Size | Author |
---|---|---|---|
#53 | 2923810-53.patch | 20.17 KB | quietone |
#53 | interdiff-47-53.txt | 1.37 KB | quietone |
#47 | interdiff-45-47.txt | 562 bytes | quietone |
#47 | 2923810-47.patch | 20.76 KB | quietone |
#45 | 2923810-45.patch | 20.44 KB | quietone |
Comments
Comment #2
phenaproximaBy committer consensus, all Migrate criticals are now straight-up critical.
Comment #3
phenaproximaThis will need a change record, as well as updates to this one.
Comment #4
dipakmdhrm CreditAttribution: dipakmdhrm as a volunteer and at QED42 commentedAfter spending some time creating patch by pulling code from #2908282-53: Throw exception for source plugins without a source_module property, I realized that it's not the only thing needed for this patch.
We need to rebuild the patch keeping the fact in mind that 'destination_module' property is only valid for migration into Drupal destinations. See #2908282-64: Throw exception for source plugins without a source_module property.
Adding links to related comments from #2908282: Throw exception for source plugins without a source_module property in Issue Summary.
Comment #5
xjm@catch, @effulgentsia, @Cottser, @lauriii and I discussed this and decided that in this case it's probably actually major. This is an important issue for making Migrate more developer- and user-friendly, but we wouldn't block marking Migrate stable on it. (We allow throwing new exceptions in minor releases especially if the code would break later anyway.)
Comment #7
Gábor HojtsyIf its not critical, it cannot be Migrate critical either. I believe we decided to harmonize these in Vienna last autumn.
Comment #8
heddnLet's get this kicked off. I'm sure there will be several failures.
Comment #10
phenaproximaLet us fix those config schema errors. This will probably still fail. But hopefully not as hard.
Comment #12
phenaproximaA little bit softer now...
Comment #14
heddnI think we are going to have to override the destination plugin manager similar to what we did for the base migration plugin manger and provide our own createInstance special snowflake that calls getDestinationModule during the constructor. It is a little later than discovery and
processDefinition()
, but at least its still pretty early in the bootstrap.Comment #15
heddnHere's a quick test to see if moving this down into
createInstance()
even makes sense. No interdiff, as I'm still playing around.Comment #16
rakesh.gectcrI have just added the interdiff, If anyone wants to look in b/w
Comment #18
heddnComment #20
heddnHere's a more real patch. Let's see how many errors we have.
Comment #22
heddnWe should now load all destination modules and test they have a destination module, similar to how we are doing that for source_module. I think, given the nature of how we are gathering things, all modules /will/ have a destination module. But we should test for that.
Comment #23
rakesh.gectcrTried adding a test, Please review and let me know, is this the right direction?
Comment #25
heddnTake a look at #2937045: source plugin source_module testing seems incomplete and that's what we'll want to do, but for destinations. But since that one is already RTBC let's just postpone this on it.
Comment #26
heddnHit the wrong status on that last save. Resetting status.
Comment #27
rakesh.gectcrApologies #23 was totally wrong patch
Comment #28
rakesh.gectcrComment #29
phenaproximaI'm unclear on why we should need to override the destination plugin manager for this functionality. Don't we only want to enforce the destination_module in the context of the migration (given a specific set of tags), not the destination itself? Isn't that why we put the source_module enforcement into the overridden migration manager, not the source plugin manager?
Comment #30
phenaproximaSelf-assigning, because I want to figure this one out.
Comment #31
phenaproximaOkay, let's try this on for size. It passes tests on my machine. Still needs a dedicated test with all providers enabled, but let's see if there are any failures to deal with first.
The hasDefinition() checks are needed in order to get the tests to pass, because in many tests, certain destinations (like entity:node) won't exist unless certain modules are enabled, thus leading to PluginNotFoundException being thrown. This leads me to agree with @heddn that we should do the source check in processDefinition(), as before, and the destination check in createInstance(). It's weird, but it might be the most appropriate path forward. We can discuss that when we have a green patch with proper tests, though.
No interdiff because I rolled this straight from a clean 8.6.x.
Comment #32
rakesh.gectcrDiscussed this on migration Maintainers meeting, assigning myself writing the test
Comment #33
rakesh.gectcrI have added the test according to the direction of #25
Looks like its going to be some code duplication. so made some changes according to that.
Do we need to add
testdestinationProvider
liketestSourceProvider
?Comment #34
phenaproximaLooks good! Thanks for writing the test. I did a little bit of refactoring to slim it down a bit, and fixed some whitespace issues.
Comment #37
rakesh.gectcrComment #38
phenaproximaThanks for those fixes -- one change, though. NullDestination's destination_module should probably be migrate, not system.
Comment #39
rakesh.gectcrThanks Adam :)
Comment #40
quietone CreditAttribution: quietone at Acro Commerce commentedI just reviewed the comments and found 1 main issues, that is the comments state that the source or destination module property must be on the annotation but isn't it really a requirement of the plugin definition?
The source_module property can be defined in the migration yml as well.
Same here, it can be defined in the migration yml.
trailing whitespace
trailing whitespace
Is it raise or throw for an exception? Just using dreditor but the 80 column wrapping looks wrong.
Hmm, 'plugin annotations' again. Are the definitions in the migration yml considered annotations?
Here there is no mention of 'annotation'. It just referring to the plugin definition which is better and accurate.
Comment #41
rakesh.gectcrComment #42
rakesh.gectcrComment #45
quietone CreditAttribution: quietone at Acro Commerce commentedJust a reroll but includes fix whitespace errors
Comment #47
quietone CreditAttribution: quietone as a volunteer commentedAdd destination_module to the color destination plugin.
Comment #48
quietone CreditAttribution: quietone as a volunteer commentedTests were added in #33
Comment #53
quietone CreditAttribution: quietone as a volunteer commentedAdd destination_module to MigrateEntityComplete.