Problem/Motivation
Calling MigrationPluginManager::getDefinitions() when the migrate_drupal module is enabled and no connection is defined for the drupal migrations, we get ConnectionNotDefinedException from the node derivers, and thus get no migrations back (even the ones that are fine). This happens because the derivers execute the node_type source plugin - with no explicit database connection configured, SqlBase attempts to use a connection named 'migrate' and if no such connection exists ConnectionNotDefinedException is thrown.
Proposed resolution
When SqlBase falls back to attempting the 'migrate' connection, the lack of such a connection should be treated as a RequirementsException rather than allowing ConnectionNotDefinedException to propagate. SqlBase should also implement RequirementsInterface, which should throw RequirementsException in this instance. And, the node derivers should check the requirements on the node_type migration and return without deriving any node migrations when the requirements are not met.
Remaining tasks
Extract the relevant parts of the patch in #2700693: [meta] Make MigratePluginManager::getDefinitions() work cleanly with migrate_drupal enabled.
User interface changes
N/A
API changes
getDatabase() will now throw RequirementsException in the absence of a configured database connection.
SqlBase and all derived source plugins will implement RequirementsInterface, and thus checkRequirements() will now be available on all such source plugins.
Data model changes
N/A
| Comment | File | Size | Author |
|---|---|---|---|
| #28 | interdiff.txt | 760 bytes | denutkarsh |
| #28 | 2830036-28.patch | 9.54 KB | denutkarsh |
| #22 | interdiff.txt | 6.12 KB | mikeryan |
| #21 | 2830036-21.patch | 9.54 KB | mikeryan |
| #21 | 2830036-21-FAIL.patch | 8.42 KB | mikeryan |
Comments
Comment #2
mikeryanComment #3
mikeryanI need to add a fail test where the node derivers blow up.
Comment #4
mikeryanHere we go.
Comment #9
mikeryanResults as expected - ready for human review.
Comment #10
mikeryanDraft change record added.
Comment #11
heddnWhat if someone for legacy reasons defined their connection 'migrate'? Why does it matter if the key exists or not? Just add the key to the requirements exception and throw it anyway.
Is this necessary for fixing this error?
Same here. This seems like a stray code cleanup.
Can we check migration plugins that use a source plugin of embed_data? That would make this more extensible for the future.
Can we check migration plugins that use a source plugin of embed_data?
Comment #12
mikeryanThe distinction being made here is that 'migrate' is an established (if undocumented) fallback in the absence of any explicit key. Thus, if no explicit key has been provided, the requirement is that there be a 'migrate' key, so if that does not exist then we've failed to meet requirements. If, however, you have provided an explicit key but neglected to actually define the connection, I think it is appropriate to allow the ConnectionNotDefinedException to propagate.
It is necessary to separate out the static::getSourcePlugin('d*_node_type') calls to check the requirements up front - having done that, it makes sense to use that result in the for loop rather than duplicate the calls.
Even better, we can check that we have none left that use a SQL-based source plugin (patch attached).
(Edit: inserted missing 'not')
Comment #13
heddnNit:
\Drupal\migrate\Plugin\migrate\source\SqlBase::classis more resilient. Or even import that thing andSqlBase::classComment #14
mikeryanDone, thanks!
Comment #15
heddnComment #17
mikeryanSimple reroll.
Comment #18
phenaproximaSelf-assigning for review.
Comment #19
phenaproximaCan this be changed to "Thrown if no source database connection is configured"?
Nit: Should be "fall back".
Let's pass the original ConnectionNotDefinedException to the RequirementsException as the previous exception (the fourth constructor argument).
What does requirements_met do? Is it documented in the plugin annotation? If not, can this be commented?
Why is this test part of Node? Seems like it should be part of Migrate.
Let's clarify this a bit: "...in the absence of a source database connection (i.e., a connection with the 'migrate' key)."
This is kind of a nit, but I don't really like that we're calling
$migration->getSourcePlugin()three times, especially since we have no need for the rest of the migration. Can we do something likearray_map(function ($migration_plugin) { return $migration_plugin->getSourcePlugin(); }, $migration_plugins)to avoid this?Comment #20
mikeryanAlso needs work because broken by #2746671: CCK field data not available for D7 taxonomy term migrations (as uncovered by brantwynn at https://www.drupal.org/node/2700693#comment-11854945).
Comment #21
mikeryanIt is documented in plugin annotation - it defaults to TRUE and gets set FALSE when requirements are not met, thus in this case if there's already a requirements failure we don't check.
It was part of the node module since we were focused on the errors generated by the node derivers. However, since we're now also dealing with taxonomy derivers as well, I've moved the tests into the existing MigrationPluginListTest in migrate.
All other @phenaproxima issues are addressed, as well as the taxonomy deriver bug (a patch without the taxonomy deriver fix is included to demonstrate that failure).
Comment #22
mikeryanComment #24
mikeryanFail patch failed as expected.
Comment #25
phenaproximaSelf-assigning for review...again.
Comment #26
phenaproximaSupernit: there should be a space after $this->pluginId. Fixable on commit.
Otherwise, I love it.
Comment #28
denutkarsh commentedSetting this issue to Needs Review. @phenaproxima I am uploading the new patch which follows the suggestion in #26.
Comment #29
denutkarsh commentedI am setting this to RTBC as set by @phenaproxima.
Comment #31
mikeryanLooks like one of those random fails, retesting...
Comment #32
hongpong commentedApplying this patch improved my luck in attempting to import via wordpress_migrate , see #2839482: Wordpress Migration UI dies with a ConnectionNotDefinedException when import starts for the gory details. I got my posts into the site correctly, if not the categories! RTBC please.
Comment #33
catchThe fix looks fine but is there somewhere we can add some docs for contrib doing similar with derivers?
Comment #34
heddnworking on that.
Comment #35
heddnAdd a CR.
Comment #37
catchCommitted/pushed to 8.3.x and cherry-picked to 8.2.x, thanks!
Comment #38
hongpong commentedThank you catch, mikeryan and everyone - I appreciate the prompt attention to this issue. Best regards.
Comment #42
mikeryanFYI - this actually was not committed to 8.2.x, look for it in 8.3.x forward only...
Comment #43
denutkarsh commentedComment #44
quietone commentedPublished change record.
Comment #45
quietone commentedPublish change record.