Problem/Motivation
We wanted to keep the LoadEntity simple and not pluginify field types so we moved the per field type processing into methods on the LoadEntity. That works OK for core but after writing a sample module for contrib (https://www.drupal.org/sandbox/benjy/2422307) it was obvious the DX isn't really good enough.
Right now as a contrib module providing a custom field type you have to hook yourself into 4 migrations to make sure everything is migrated appropriately. Any less and things blow up.
Proposed resolution
Add a new plugin type and call that from LoadEntity for per CCK field type processing.
Remaining tasks
Reviews
User interface changes
n/a
API changes
New plugin type for adding support for custom field types in migrate.
Comment | File | Size | Author |
---|---|---|---|
#26 | interdiff.txt | 1.29 KB | benjy |
#26 | 2423435-26.patch | 14.31 KB | benjy |
#23 | interdiff.txt | 2.1 KB | benjy |
#23 | 2423435-23.patch | 14.18 KB | benjy |
#20 | interdiff.txt | 4.93 KB | benjy |
Comments
Comment #1
chx CreditAttribution: chx commentedThis is not a migrate thing; it's a migrate_drupal thing.
Comment #2
benjy CreditAttribution: benjy commentedOK, first version of the patch, leaving as postponed because it depends on #2422227: Add setProcessOfProperty to Migration entity
Few notes:
* I added methods like isFieldMigration(), isFieldWigetMigration to MigrationStorage so we could easily add the D7 migrations when they come about. We could also fire a drupal_alter here to allow other modules to add migrations in case D5 migrations ever come in contrib. Or, they could override the storage controller.
* Core isn't using the new plugin type yet. If we're happy on the general approach i'll convert a few field types. It does work though, I tested with a contrib port of iFrame: https://www.drupal.org/sandbox/benjy/2422307
Comment #3
benjy CreditAttribution: benjy commentedAdded a migration id to plugin method map instead of the individual methods.
Comment #5
benjy CreditAttribution: benjy commentedFurther clean-up, not looked at the fails yet.
Comment #7
benjy CreditAttribution: benjy commentedNasty little problem where the source plugin's config gets cached because the storage controller calls getSourcePlugin() and then if anything else tries to set the source config after the migration has been loaded it is never used.
The attached patch shows the problem but I'm not convinced it's the correct approach?
Another approach could be to add a setSourceConfig() method onto the migration and then invalidate the source plugin cache from there? I don't much like $entity->set('source', $source) anyway.
Comment #9
benjy CreditAttribution: benjy commentedThe problem was trying to access fieldData() for the migration with the load plugin attached when we only want that for the bundle migration.
We could have just checked for that migration like below but I decided to checkout against $migration->load
Comment #10
chx CreditAttribution: chx commentedWouldn't
!$migration->get('load')
suffice? Also, I am surprised that superflous processing causes tests to break. Of course we shouldn't do superflous processing so this needs to stay but isn't processing a no-op when applied to migrations we do not need/want to process?Comment #11
benjy CreditAttribution: benjy commentedYes, that would also work.
It isn't the actual processing that causes the error. It's an undefined index CckFieldValues::fieldData() when $this->configuration['bundle'] is not set. Eg for the top level d6_cck_field_values migration.
Comment #12
benjy CreditAttribution: benjy commentedSwitched to using getDerivativeId() instead of checking the load plugin array.
Added a setSourceConfig method which is only really used in tests and invalidated the sourcePlugin cache.
Comment #13
benjy CreditAttribution: benjy commentedThis is nearly ready but postponed on: #2422227: Add setProcessOfProperty to Migration entity
Just need to convert some core field types to use this.
Comment #14
benjy CreditAttribution: benjy commentedOK, this is unblocked now. Re-rolled against HEAD.
Comment #15
chx CreditAttribution: chx commentedDo we perhaps want to override set() instead? I think we want to.
Comment #16
benjy CreditAttribution: benjy commentedYes I like that, I think it's better than requiring another public method on the entity.
Comment #17
benjy CreditAttribution: benjy commentedComment #18
amateescu CreditAttribution: amateescu commentedHere's a small review, nothing alarming stands out :)
Afaik, we have to include the leading slash everywhere we use a class or interface name as a string.
'successfully migration' sounds a bit weird :)
We can save the use of
/** @var ... */
by using\Drupal\migrate\Entity\Migration[]
instead of array in the docblock.Looks like we can move these at the top of the method to save some calls in the foreach.
Are we missing a d6_field_instance here?
How about renaming everything to use 'formatter' instead of 'display'?
Comment #19
amateescu CreditAttribution: amateescu commentedAlso, related to the last point (#18.6), it seems that we're missing a
getFieldWidgetMap()
method :)Comment #20
benjy CreditAttribution: benjy commentedComment #21
amateescu CreditAttribution: amateescu commentedSo, I looked at d6_field_instance_widget_settings and that seems to hardcode the 1:1 mapping for *known widgets*, but isn't it possible, at least theoretically, that a contrib module provided two widgets in D6 and combined them into a single one (with more options) in D8?
Comment #22
benjy CreditAttribution: benjy commentedSure, in that use you'd just add two entries to this same map, same as we've done for datetime_default
I added getFieldFormatterMap() to try give a slightly nicer DX. It allows contrib authors to skip implementing processFieldFormatter() and simply allows them return an array from getFieldFormatterMap() so the parent can handle updating the process array.
processFieldWidget() also has a default implementation in CckFieldPluginBase that sets a sane default based on your module name.
Nothing stops contrib modules doing what they want in processFieldWidget() and processFieldFormatter() and in that case they could ignore the return value from getFieldFormatterMap() altogether.
Maybe i've just confused things by adding getFieldFormatterMap(), we could change how that works?
Comment #23
benjy CreditAttribution: benjy commentedOK, discussed this on IRC and we've now added a getFieldWidgetMap() to be consistent. I should also point out the lack of tests here, we'll be using the API in core for the link field etc which will give us implicit coverage, created a follow-up #2448501: Use the migrate cckfield plugin type for the link field
Comment #24
amateescu CreditAttribution: amateescu commentedAwesome, looks good to me now :)
Comment #25
alexpottNot used.
Missing docblocks for $field_name and $data
Comment #26
benjy CreditAttribution: benjy commentedFixed.
Comment #27
chx CreditAttribution: chx at MongoDB commentedAlex's concerns are addressed.
Comment #28
alexpottRe #23 I think the lack of explicit test coverage might come back to bite us but maybe when #2448501: Use the migrate cckfield plugin type for the link field lands it will be more obvious how to add explicit coverage.
Committed 04a17f8 and pushed to 8.0.x. Thanks!