Migrate on D7 has the concept of migration groups. These have two primary uses:
1. Easily manipulating a set of related migrations together - e.g., drush migrate-import --group=commercesite
2. Storage of configuration shared among related migrations, most commonly source information - migrate_d2d stores database connection info in the group, wordpress_migrate stores the location of the source WXR file, etc.
This will be a good thing to build into the core framework in D8, and to have migrate_drupal group the migrations it manages to separate them from any other migrations that might be instantiated in the environment.
In D7, groups were maintained in the migrate_group table, and each individual migration had a group machine_name in the migrate_status table pointing to its group. I'm not sure that's necessarily best in D8 - perhaps it would be better for each group to have a list of member migrations? This would enable a given migration to belong to more than one group, although the concrete use case for doing that escapes me at the moment...
Comment | File | Size | Author |
---|---|---|---|
#18 | implement_migration-2202511-18.patch | 44.86 KB | hussainweb |
#16 | 2202511-16.patch | 44.81 KB | benjy |
Comments
Comment #1
moshe weitzman CreditAttribution: moshe weitzman commentedViews config entities have a 'tag'. Might be useful to mimic that here.
Comment #2
benjy CreditAttribution: benjy commentedI'm going to have a go at this in the next week to assist with D7 migrations.
Comment #3
benjy CreditAttribution: benjy commentedFirst attempt at this attached.
Few things to discuss:
Comment #4
hussainwebDrupal 7's migrate only allows a migration to be in a single group but it would be a cool thing to support multiple groups. I am just wondering how many use cases there would be for that.
How about just
groups
?It sounds like a good idea.
Comment #5
hussainwebTo elaborate on the second point: None of the other keys are prefixed. We could keep it consistent and call it just
groups
without any prefix.About point 2 in the issue:
Should this be implemented as a new config entity, perhaps? If we go this route, then maybe we should follow D7 migrate's approach with using a machine name for group (also would help the drush command line) and more details in the group's yml.
Comment #6
benjy CreditAttribution: benjy commented1. Many of the D7 migrations are going to be quite similar to the D6 ones.
2. Not exactly true, we have migration_dependencies because "dependencies" conflicted with a parent property.
Yeah, I don't mind the idea of using a machine name instead, that doesn't have to be 1 to 1 with a new config entity. Not sure about the new config entity, I guess we'd have to do that if we want to be able to store configuration in the group as the issue summary states?
Comment #7
chx CreditAttribution: chx commentedThis is an awesome first step that can iterate later. If we need to change from Drupal 6 to drupal_6 that's fine because in HEAD
grep -- ' - Drupal 6' *
in the config directory comes back empty so search/replacing that will target these only.Comment #8
benjy CreditAttribution: benjy commentedAdded a follow-up for the group configuration stuff. #2302307: Support shared configuration between migrate groups
Comment #9
hussainweb@benjy: I see your point about migration_dependencies. However, I see to remember an issue that removed prefixes from all (or most of) the properties. Is there a reason we cannot use
groups
? I am just trying it out to see if there are any obvious failures.Comment #10
benjy CreditAttribution: benjy commentedThe reason I didn't want to use groups is because it's quite generic and if we ever decide to add "groups" in the base config entity like we did with "dependencies" we'd need an API change.
plus, migration_groups is more descriptive, I think I prefer it.
+1 for #3.
Comment #11
hussainweb@benjy: Thanks for the explanation. I looked back at the YML files and do see your point. I think
groups
is descriptive enough, especially when it lives in the migration yml file but the point about base config entity is more important.Comment #12
alexpottIs #3 rtbc or #9? I think #3 - can the real rtbc patch be uploaded last since this is the patch that the rtbc retest get's run against. Thanks.
Comment #13
hussainwebUploading patch from #3 again.
Comment #15
benjy CreditAttribution: benjy commentedBack to RTBC since the last patch isn't the patch that was RTBC
Comment #16
benjy CreditAttribution: benjy commentedO, sorry, cross post. Re-rolled after conflict with #2302261: Typo in migrate.schema.yml
Comment #18
hussainwebFixing conflicts from #2292821: Use widget for comment subject field. It was just a small conflict and I guess it can go back to RTBC as soon as tests pass.
Comment #19
hussainwebBack to RTBC as per #7.
Comment #20
alexpottCommitted 7a42da2 and pushed to 8.x. Thanks!