Originally a followup to #2543536: [meta] Reduce/remove tight coupling of migration classes - the scope here has narrowed to fixing the gross hackery of md_entity, and the original issue is now covered by #2822021: Reduce/remove tight coupling of migration destination plugins.
Problem/Motivation
migrate_drupal's EntityFieldStorageConfig does the following:
public function calculateDependencies() {
$this->dependencies = parent::calculateDependencies();
// Add a dependency on the module that provides the field type using the
// source plugin configuration.
$source_configuration = $this->migration->get('source');
if (isset($source_configuration['constants']['type'])) {
$field_type = $this->fieldTypePluginManager->getDefinition($source_configuration['constants']['type']);
$this->addDependency('module', $field_type['provider']);
}
return $this->dependencies;
}
The destination plugin is reaching through the migration to get the 'type' out of the source plugin's configuration, so it can determine what destination module it depends on. This is a complete WTF - I'm sure it must be at least tangentially related to #2507607: [META] Replace Cthulhu-forsaken load plugins with migration builders, but even if that effort doesn't remove this, we must kill it dead.
Proposed resolution
Fix the EntityFieldStorageConfig WTF.
Remaining tasks
User interface changes
N/A
API changes
None.
Data model changes
None anticipated.
| Comment | File | Size | Author |
|---|---|---|---|
| #69 | interdiff.txt | 643 bytes | mikeryan |
| #69 | remove_the_md_entity-2543568-69.patch | 5.08 KB | mikeryan |
| #64 | interdiff.txt | 4.78 KB | mikeryan |
| #64 | remove_the_md_entity-2543568-64.patch | 5.12 KB | mikeryan |
| #59 | interdiff-2543568-56-59.txt | 4.43 KB | chipway |
Comments
Comment #1
mikeryanComment #2
mikeryanThe calculateDependencies() thing was added in #2460529: Migrations need to use the configuration entity dependency system. Looking at that issue, and how this plugin is used, I believe #2533886: [meta] Move module-specific migration support into the particular modules supported should obsolete the entire plugin - once all migrations are defined within their destination modules, this hack to enforce the dependency on that module will no longer be necessary.
Comment #3
mikeryanDigging in a little more deeply to identify where EntityFieldStorageConfig::calculateDependencies() has an effect:
I think for use cases #2 & #3, forcing the dependency directly is less of a hack than what we have going on in EntityFieldStorageConfig::calculateDependencies now.
Comment #4
mikeryanI've started playing around with this, and the migration coupling seems to have similar defense mechanisms to Alien - you don't dare remove it. Posting what I did so far, will pick it up again at some other point.
Comment #5
benjy commentedI'm surprised we can remove this, although it's good. Is this because of the builders?
Comment #6
mikeryanShould be doable now that load plugins are gone.
Comment #7
mikeryanI do believe I've removed the tumor without killing the patient...
Comment #9
mikeryanOops, missed one.
Comment #12
mikeryanRerolled for changed EntityComment.php.
Comment #13
quietone commentedNo longer applies.
Comment #14
quietone commentedInterdiff fails.
Comment #16
quietone commentedHah, I think I submitted while that webchick was doing a bunch of commits for migrate.
This one should be ok. And, as before the interdiff fails.
Comment #18
quietone commentedAnd again.
Comment #20
quietone commentedYep, another commit was made, thus the error. That is fixed in the attached.
Comment #22
quietone commentedAnother reroll.
Comment #24
quietone commentedReroll for 8.1.x
Comment #25
mikeryanNew work (tasks/features) should be targeted to the 8.2.x branch.
Writing a change record may be a Novice task...
Comment #26
quietone commentedOk, I'm not much of a word smith and I've not done a change record but let's keep this moving. How is this for a start?
Title: $migration parameter removed from destination plugin declaration
Project: Drupal Core
Introduced in branch/version: 8.2.x
Issues: 2543568
Description: The migration destination plugins used to receive the $migration parameter, allowing full access to the migration configuration. This coupling has been removed so that the destination plugins are independent.
What else is needed?
Comment #27
chx commentedThis:
And if I were doing this (I am not) I would postpone this on #2695297: Refactor EntityFile and use process plugins instead.
Comment #28
mikeryanLet's deprecate md_entity:field_storage_config rather than remove it entirely.
Comment #29
heddnRemoving novice tag. Writing meaningful change notices are usually harder than you think. I did a review of the code. Nothing jumped out at me that seems strange. We have plenty of test coverage in this area. So I think this is RTBC.
Wise to keep it around. Who know who is doing what at this point. Providing an @deprecated is about the best we can do.
Comment #30
alexpottLooks like this shouldn't be removed.
Given #3 I'm surprised we can remove this without adding any dependencies anywhere. How come this is possible?
Comment #31
Anonymous (not verified) commentedAdding the return statement back to the annotation.
Comment #32
Anonymous (not verified) commentedComment #33
mikeryanFor migrations of field storage configuration from Drupal, this takes the source configuration constants/type (the name of the field type being created in the migration) and makes the module of the same name as the field type a dependency. Things wrong with this:
So, we really should get rid of this md_entity:field_storage_config nonsense. But, yes, there should be some way to add dependencies on the field module. The most direct way to do that would be to simply make them explicit in the destination configuration (where they belong). Say, for user_picture_field, replace
with
Thoughts?
Comment #34
mikeryanLet's try that.
Comment #36
mikeryanComment #37
mikeryanChange record added.
Comment #38
phenaproximaSince this now extends DefaultPluginManager, why do we need to bother copying and pasting the factory logic here? DefaultPluginManager uses ContainerFactory by default, which is aware of ContainerFactoryPluginInterface.
I imagine we'll probably remove this *before* D9. Can we s/in/before?
Where are these used? It seems to me that we should have a calculateDependencies() implementation on DestinationBase that actually makes use of this configuration.
Comment #39
mikeryanTurned out we needed a reroll anyway with recent commits - no interdiff, because it got confused...
Well, the assumption was to retain the MigratePluginManager stuff here - but, the comment "A specific createInstance method is necessary to pass the migration on" suggests that's no longer needed, so let's see how we do without createInstance()...
Done.
'dependencies' is part of the base plugin system. The point of calculateDependencies() would be to automatically calculate them, but, per #33 above, the attempt previously being made to calculate them was misguided - it just happened to get lucky with these cases provided in core, but would break in the more general case, so it's best to simply make the dependencies explicit.
Comment #41
mikeryanAre those new tests? Hard to see how they would have been passing previously... Anyway, fixed those up...
Comment #42
neclimdulre #41: newish but no, blame says they where both part of the d6 translation patch. Your patch changes the EntityContentBase constructor so it makes sense it would have to fix the tests instantiating it directly. That or keep the argument and ignore the value depending on how we want to handle BC.
Comment #43
phenaproximaWhat is $plugin_interface_map? Doesn't seem to be defined in this function, or as a parameter.
Apart from that, I think this is RTBC.
Comment #44
mikeryanAs with createInstance(), when switching from extending MigratePluginManager to extending DefaultPluginManager, this was copied from MigratePluginManager on the assumption it was still needed. So, let's take it out (note: MigratePluginManager also has this bit of code which seems to do nothing).
Comment #45
phenaproximaPreemptively RTBC. I see no reason why Drupal CI won't pass this.
Comment #46
alexpottGiven that migrate is still not out of alpha what is the advantage of deprecating this and not removing?
Comment #47
alexpottNice diff stat btw.
Comment #48
mikeryanWe'd still like to minimize disruption where we can. There may be contrib field modules which used md_entity for their migrations paths - probably more commonly, people who've used the practice of doing migrate-upgrade --configure-only will still have md_entity references in their generated migrations, which will bite them if we remove it entirely immediately.
Comment #49
alexpottAnd they won't need to make any changes if we leave it as is?
Comment #50
chx commentedIn particular
For what? These plugins are thin wrappers around perfectly usable APIs on their own. I fail to see how a migration destination plugin could be used for anything else but being the glue between Row and whatever CRUD API it wraps.
Comment #51
mikeryanIf their field migration depends on a module other than the one defining the migration, then they should add an explicit dependency to the migration. So - I see what you're saying, they might as well change the destination plugin while they're at it. At least some, though, would have no problem with the BC layer present, so it seems nice to save them the trouble of "why am I suddenly getting errors about md_entity" if it's simple. I'm not wedded to that though, I can remove the BC layer if you prefer.
As for removing the Migration argument, when opened the idea here was simple API cleanup - there is no reason the destination plugin should need the migration plugin, and it only existed to support this ugly hack, so we might as well remove it. A year ago I would have fought to get it in, but unfortunately it didn't get RTBC'd until after 8.2.0-rc - given that we're trying very hard to stabilize the API, and this particular BC break is more about API cleanliness than practical benefits, it's not worth wasting more time arguing over it. The attached patch fixes the md_entity WTF without removing the migration parameter from the migration plugins (I did keep removal of the optional migration parameter from the fields() method - that makes no sense whatsoever).
Comment #52
mikeryanBTW, no interdiff because the interdiff is bigger than the new patch...
Comment #53
heddnIs this BC breaking still?
Comment #54
mikeryanNo - see #51.
Comment #55
claudiu.cristeaIt cannot be "removed before Drupal 9.0.x" but it can be:
OR
Comment #56
imiksuLets go with "removed in Drupal 9.0.x".
Comment #57
claudiu.cristeaComment #58
mikeryanNow that I'm explicitly narrowing the scope, I recognize that removing the unused $migration parameter from fields() is out-of-scope here. Taking those changes out is a novice task - if no one picks it up in a day or so I'll do it myself.
Comment #59
chipway commentedApplied mikeryan comment #58.
Comment #60
heddnAssigning to myself to review. Reviews from others is fine in the mean time.
Comment #61
heddnThere are no more md_entity mentions in all of core, except for the BC layer. The migration is no longer passed into fields. Do we need to add a test of the BC layer? Me thinks no. Ready to go.
Comment #62
heddnun-assigning.
Comment #63
alexpottDoes this need a new issue then? Rather than removing the comment.
This is the only code that might result in different behaviour. I'm not sure why we are emptying this class.
I think removing this would break things. Since it'll now do:
Which wouldn't return 'field_storage_config'. Again I just think we should be deprecating the class and not actually changing it. If we change it we definitely need a test to prove we've not broken it.
Comment #64
mikeryanNew patch deprecates the parameter (and documents that it is unused).
It's kind of the whole point to remove this hack, which pretty much only works by accident... But, if simple deprecation is the easiest path forward, let's do that.
Comment #65
heddnAssigning to myself for a final review.
Comment #66
heddnI can see what we are trying to do here, but I don't think we can mark a parameter deprecated via an annotation. But I also cannot find any docs on how or if this is allowed or possible.
Except for that small nit, this is good to go. Anyone know?
Comment #67
heddnComment #68
mikeryanI'll remove the annotation and expand the comment.
Comment #69
mikeryanComment #70
heddnRandom test failure against 8.3.x. All feedback addressed. Back to RTBC.
Comment #72
catchCommitted/pushed to 8.3.x and cherry-picked to 8.2.x. Thanks!
Comment #74
xjmI think this may have broken HEAD.
Comment #75
xjmIt was actually #2746671: CCK field data not available for D7 taxonomy term migrations so I left this issue alone. :)