Early Bird Registration for DrupalCon Portland 2024 is open! Register by 23:59 PST on 31 March 2024, to get $100 off your ticket.
Problem/Motivation
Follow up to #3198608: trackLastImported YAML key to enable tracking the last import date of a row is undocumented and confusing. and also much needed.
Document the available definition properties of a migration defined in a yml.
Steps to reproduce
Go to class Migration and see no list or explanation of the properties.
Proposed resolution
Write the documentation
Remaining tasks
Patch
Review
Commit
User interface changes
N/A
API changes
N/A
Data model changes
N/A
Release notes snippet
N/A
Comment | File | Size | Author |
---|---|---|---|
#14 | 3226401-14.patch | 3.44 KB | quietone |
#14 | interdiff-13-14.txt | 924 bytes | quietone |
#13 | 3226401-13.patch | 3.05 KB | quietone |
#13 | interdiff-7-13.txt | 656 bytes | quietone |
#7 | 3226401-7.patch | 3.05 KB | quietone |
Comments
Comment #2
quietone CreditAttribution: quietone as a volunteer commentedOK, here is a start. Reviews welcome. Not running test because this is a doc only change.
Comment #3
huzookaWhat if we change also this line? I'd rather start this sentence with "A migration plugin instance..." It isn't crucial, but I've seen a lot of confusion already in regards of what is a plugin class, what is a plugin annotation, what is about a plugin definition, and what is an actual instance.
It would be nice to see some notes here about the migration yaml files (see the issue title). Something like
"Migration yml files in migrations subdirectories are plugin annotations written in Yaml format (they're are almost complete plugin definitions)..."
It's a label, not a description. See Migration::label:
Can this be somehow formulated as "process pipelines of the destination properties"?
Afaik nothing enforces that optional dependencies MUST be executed before running an actual migration.
See e.g. MigrationPluginManager@L191
$required_dependency_graph = (new Graph($required_dependency_graph))->searchAndSort();
.TODO I guess.
Example with all keys misses
provider
.Comment #4
quietone CreditAttribution: quietone as a volunteer commented@huzooka, thanks!
This patch responds to all the points in #3.
Comment #5
joachim CreditAttribution: joachim at Factorial GmbH, TrainingCloud commentedLooks good, but a few typos:
I don't think 'annotations' is the right word here.
Should simply say 'Migrations are plugins defined ...'
Sentence doesn't read properly.
should be 'listing the migrationS'
missing plural 's' here too
'executeD'
I've been using 'dependencies' for this, probably because I saw it once elsewhere. Does that work too, or have I been doing this pointlessly?
Comment #6
huzooka@joachim, with respect:
dependencies
isn't a thing in in Migrate API's migration definitions/annotations (name they as you prefer).You're mixing these with Migrate Plus migration configuration entities.
And regarding to your #1 point: the word "plugin" is ambiguous. If you just say "plugin", then we don't know what do you refer. It might be a (base) plugin definition described by the yaml files (or the definition of a derivative), or a plugin instance created by the manager.
Comment #7
quietone CreditAttribution: quietone as a volunteer commentedThanks for the reviews.
Changes for #6 2-6.
This sentence needs work and my brain is not up to it at the moment. Using YAML is only one way to create a migration plugin.
"Migrations are plugin annotations defined using YAML format and placed in the directory MODULENAME/migrations."
Comment #8
joachim CreditAttribution: joachim at Factorial GmbH, TrainingCloud commented> @joachim, with respect: dependencies isn't a thing in in Migrate API's migration definitions/annotations (name they as you prefer).
Yup, that's what I meant -- I've been using these incorrectly as they have no effect! Must be a leftover from config migrations. Thanks for the clarification.
> "Migrations are plugin annotations defined using YAML format and placed in the directory MODULENAME/migrations."
Annotations are metadata in source code comments. In Drupal, we use Doctrine Annotations, and they're the '@Foo{}' type syntax in PHP comments, and we use them to put plugin definitions in the same code file as the PHP class for that plugin.
The plugin definition is the annotation. But for a YAML plugin, the plugin definition is the YAML. The definition is an array of data. The *format* of that definition is either an annotation, or a YAML file. And the plugin itself is an instantiated object of a particular class, which holds the definition array.
> Using YAML is only one way to create a migration plugin.
Do you mean using plugin derivers?
I would put:
> Migrations are plugins defined as YAML files and placed in the directory MODULENAME/migrations.
and if we want to also mention using derivatives, we can do so further down.
We could link to the API topic on plugins (https://api.drupal.org/api/drupal/core%21core.api.php/group/plugin_api/9...) but that only mentions annotation plugins and not YAML, so it's not that relevant.
Comment #9
huzookaImho this is not true for migrations. Think about fieldable entity migrations: their definition contains the process pipelines of their fields as well, while their YAML file does not. And these field processes changing per derivative. If you get the instance of
d7_node_complete:article
, and check its plugin definition, it will contain lot more then the corresponding (base)d7_node_complete.yml
file.Comment #10
joachim CreditAttribution: joachim at Factorial GmbH, TrainingCloud commentedSo is provider actually a single-valued property? In which case, does it work the same as the provider in other plugins, where if it's not specified, it defaults to the module where the plugin is defined?
Comment #11
joachim CreditAttribution: joachim at Factorial GmbH, TrainingCloud commented> Imho this is not true for migrations. Think about fieldable entity migrations: their definition contains the process pipelines of their fields as well, while their YAML file does not. And these field processes changing per derivative. If you get the instance of d7_node_complete:article, and check its plugin definition, it will contain lot more then the corresponding (base) d7_node_complete.yml file.
That's true for all types of plugin. Derivatives change the basic definition.
I'm not sure we're actually arguing the same point here! I'm merely saying that the word 'annotation' refers to a particular thing and doesn't apply in the context of migration plugins.
Comment #12
quietone CreditAttribution: quietone as a volunteer commentedAdding a note here to consider adding here or elsewhere the 'translations' property that can be used on the destination.
Comment #13
quietone CreditAttribution: quietone as a volunteer commented#10. Yes. I got mixed up. It is the source plugins that can have multiple providers, See \Drupal\migrate\Plugin\Discovery\AnnotatedClassDiscoveryAutomatedProviders.
In the hope of making progress, I have changed the second paragraph to use the terminology from \Drupal\migrate\Plugin\MigrationPluginManager::getDiscovery where the migration yaml files are referred to as 'configurations'.
Not running tests because this is documentation only.
Comment #14
quietone CreditAttribution: quietone as a volunteer commentedIn regard to #12, I added more @see directing to the destination plugins and source plugins in migrate module. I can't think of anything else to add here.
Comment #15
joachim CreditAttribution: joachim at Factorial GmbH commentedLGTM!
Comment #18
SpokjeBack to RTBC per #15 after #3255836: Test fails due to Composer 2.2 solved the unrelated test failure.
Comment #19
alexpottCommitted and pushed ce85149c336 to 10.0.x and 85723106f69 to 9.4.x and 372a1fe0b78 to 9.3.x. Thanks!
Backported to 9.3.x as a docs improvement.