In #2752335: Properly integrate configuration-entity-based migrations with the core plugin manager, we should be removing (or deprecating) the configuration-entity-specific plugin manager - migrate_tools should then use the core plugin manager directly.
| Comment | File | Size | Author |
|---|---|---|---|
| #10 | _bash__and_Use_the_core_plugin_manager.png | 49.01 KB | joelpittet |
| #9 | use_the_core_plugin-2795447-9-3.x.patch | 373 bytes | mikeryan |
| #9 | use_the_core_plugin-2795447-9.patch | 2.29 KB | mikeryan |
| #3 | use_the_core_plugin-2795447-3.patch | 1.92 KB | mikeryan |
Comments
Comment #2
mikeryanComment #3
mikeryanThis patch should work with #2752335: Properly integrate configuration-entity-based migrations with the core plugin manager and the upcoming patch at #2700693: [meta] Make MigratePluginManager::getDefinitions() work cleanly with migrate_drupal enabled to at least make all this a little bit better.
Comment #4
mikeryanThe core patch to use with this is #2830036: MigrationPluginManager::getDefinitions() blows up in node derivers.
Comment #5
joelpittetI gave this a try with those other patches well not the meta but the fallback #2830031: Fix SqlBase fallback priorities and document them.
It through a lot of errors on
drush msand showed a lot of migrations that may have been from tests.Example output:
Comment #6
joelpittetOh but without this, I'm getting "Migration upgrade_d6_field_formatter_settings does not exist" which is weird because it's in the status list... so this seems like the right thing to do. How can I help make this work?
Comment #7
mikeryanThe necessary core support to do this is in Drupal 8.3.x - so, I've opened an 8.x-4.x branch which will be compatible with core 8.3.x and above, and the 8.x-3.x will be only compatible with 8.2.x.
Comment #8
mikeryanComment #9
mikeryanReady to go!
Comment #10
joelpittetStill having those issues I mentioned in #5:

Comment #11
mikeryan@joelpettit: How/where are you defining the "cs6" database connection? Are you sure it's pointing at a full Drupal 6 database?
How did you get your migrations set up? The
drush msoutput is showing the raw migration IDs, but in #5 you had a message referencing upgrade_d6_field_formatter_settings, which would have been created by adrush migrate-upgrade --configure-only. I would expect all of your D6 migrations to be prefixed with 'upgrade_', or none of them.I guess, in other words... steps to reproduce?
Comment #12
joelpittet@mikerayn;) I'm sure it's pointing to the full database, cs6 is the database name I chose as I work for Computer Science and it's D6, I defined it in the settings.local.php. I've been using it for a while now to migrate. I built the migration using migrate-upgrade config only like this tutorial plus some custom class overrides and tweaks: https://drupalize.me/blog/201604/custom-drupal-drupal-migrations-migrate...
All the migrations are in a custom module's config/install folder. If I don't use this patch then I don't get that kind of
drush msoutput.Comment #13
mikeryan@joelpettit: I see that the message is for "cs6.node_revision", but in D6 the revision table name was "node_revisions" - it seems like somehow you've picked up at least one D7 migration. I also note that your migrations are in the 'default' group, where it should be 'migrate_drupal_6'.
I'm not able to reproduce this problem - here are the steps I took to try:
and it comes out fine:
Comment #14
mikeryan@joelpettit: You're not setting a 'migrate' connection in settings.php, are you? That's going to cause migrations not relevant to your use case to get instantiated. That 'migrate' fallback is very problematic and I strongly recommend against using it.
Comment #15
joelpittetSome possible relevant config:
Am I doing what you said in #14? I don't totally understand, but kinda sounds like what I'm doing...
Comment #16
mikeryanDon't do that!
As documented in the patch at #2830031: Fix SqlBase fallback priorities and document them:
Having a hard-coded global fallback was always a bad idea, and it leads into situations like this. If that global 'migrate' connection is defined, it is present for *all* the migrations in core, both those that are relevant to your specific D6 migration, and those that are not, so all those migrations will appear (and get in the way) This wasn't seen in migrate_plus/migrate_tools before, because it was only using configuration entities to define plugins, thus only the migrations that had been configured with a database connection. It's important, however, for the tools to support "regular" migration plugins (defined in the migrations or migration_templates directories of your module), and also having the two plugin managers has caused any number of subtle buggy interactions. So, while it was never a good idea to use the default 'migrate' connection, doing that becomes more painful with this patch.
It's understandable why people did it before - if they ended up trying to use migrations which were missing a configured database connection, they'd get an error along the lines of "Missing 'migrate' database connection" and naturally think they should define such a connection. With the recent changes to SqlBase in core, though, that misleading message should be much less likely to occur, and hopefully we can wean people off using the 'migrate' connection.
Comment #17
joelpittetThe database connection made sense to me without documentation. Though I appreciate that I may not be doing it the way intended, it's a bit too convenient to make that mistake and likely documented like the above tutorial to do it that way.
I'm not sure what a
database_state_keyis, so I'll have to go read and understand what it's to do.Comment #18
joelpittetOk I rebuilt the migrations as you indicated without the migrate database key and it looks better but it's still showing unexpected groups. Is it ok to be showing the 'Default' group below?
Comment #19
mikeryanUnfortunately, I haven't figured out a way to keep those extra migrations under "default" from showing up: #2700693-37: [meta] Make MigratePluginManager::getDefinitions() work cleanly with migrate_drupal enabled. Please ignore them for now.
Comment #20
joelpittetThanks, glad to know I'm not the only one:)
Comment #22
mikeryanAt long last, committed!