Problem/Motivation

With migrations based on the migrate_plus migration configuration entity, a failure like the following can happen when you are using a 'migration' process plugin:

exception 'Symfony\Component\DependencyInjection\Exception\ServiceNotFoundException' with message 'You have requested a non-existent   [error]
service "plugin.manager.migrate.cckfield". Did you mean one of these: "plugin.manager.migration", "plugin.manager.migrate.source",
"plugin.manager.migrate.process", "plugin.manager.migrate.destination", "plugin.manager.migrate.id_map", "plugin.manager.image.effect",
"plugin.manager.views.field"?' in /Users/mikeryan/Sites/8.2.x/core/lib/Drupal/Component/DependencyInjection/Container.php:157
Stack trace:
#0 /Users/mikeryan/Sites/8.2.x/core/modules/node/src/Plugin/migrate/D6NodeDeriver.php(59):
Drupal\Component\DependencyInjection\Container->get('plugin.manager....')
#1 /Users/mikeryan/Sites/8.2.x/core/lib/Drupal/Core/Plugin/Discovery/ContainerDerivativeDiscoveryDecorator.php(20):
Drupal\node\Plugin\migrate\D6NodeDeriver::create(Object(Drupal\Core\DependencyInjection\Container), 'd6_node')
#2 /Users/mikeryan/Sites/8.2.x/core/lib/Drupal/Component/Plugin/Discovery/DerivativeDiscoveryDecorator.php(98):
Drupal\Core\Plugin\Discovery\ContainerDerivativeDiscoveryDecorator->getDeriver('d6_node', Array)
#3 /Users/mikeryan/Sites/8.2.x/core/lib/Drupal/Component/Plugin/Discovery/DerivativeDiscoveryDecorator.php(86):
Drupal\Component\Plugin\Discovery\DerivativeDiscoveryDecorator->getDerivatives(Array)
#4 /Users/mikeryan/Sites/8.2.x/core/lib/Drupal/Core/Plugin/DefaultPluginManager.php(274):
Drupal\Component\Plugin\Discovery\DerivativeDiscoveryDecorator->getDefinitions()
#5 /Users/mikeryan/Sites/8.2.x/core/lib/Drupal/Core/Plugin/DefaultPluginManager.php(174):
Drupal\Core\Plugin\DefaultPluginManager->findDefinitions()
#6 /Users/mikeryan/Sites/8.2.x/core/modules/migrate/src/Plugin/MigrationPluginManager.php(141):
Drupal\Core\Plugin\DefaultPluginManager->getDefinitions()
#7 /Users/mikeryan/Sites/8.2.x/core/modules/migrate/src/Plugin/MigrationPluginManager.php(94):
Drupal\migrate\Plugin\MigrationPluginManager->expandPluginIds(Array)
#8 /Users/mikeryan/Sites/8.2.x/core/modules/migrate/src/Plugin/migrate/process/Migration.php(81):
Drupal\migrate\Plugin\MigrationPluginManager->createInstances(Array)
#9 /Users/mikeryan/Sites/8.2.x/core/modules/migrate/src/MigrateExecutable.php(392):
Drupal\migrate\Plugin\migrate\process\Migration->transform('1', Object(Drupal\migrate_tools\MigrateExecutable),
Object(Drupal\migrate\Row), 'parent')

What's happening is that the node module's D6NodeDeriver plugin (which expects to be invoked in the context of a migrate_drupal-based migration) is being invoked when the migration process plugin calls the core MigrationPluginManager to createInstances() of the migration(s) being referenced. This problem is represented in the core issue queue by #2560795: Source plugins have a hidden dependency on migrate_drupal and #2700693: [meta] Make MigratePluginManager::getDefinitions() work cleanly with migrate_drupal enabled - I had thought that using our own plugin manager with configuration entities would shield us from the consequences of those issues, but apparently not.

I'm opening this issue in migrate_plus since people will see the problem manifested here, and I'm not positive resolving those core issues will necessarily be enough to fully resolve our manifestations. Also, perhaps we can find a way to work around it in contrib until core is fixed? Can we do something with our plugin manager (like find a way to plug it in as the core plugin.manager.migration service without breaking the core upgrade path)?

Proposed resolution

  1. Fix the core issues.
  2. ?

Remaining tasks

User interface changes

N/A

API changes

Data model changes

CommentFileSizeAuthor
#5 you_have_requested_a-2729665-5.patch504 bytesmikeryan
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

mikeryan created an issue. See original summary.

mikeryan’s picture

mikeryan
he/him
Primary language English
Location Murphysboro, IL, USA
CreditAttribution: mikeryan at Virtuoso Performance commented

I should note that, for example, migrate_example breaks now like this but was working fine not very long ago - I suspect there was a recent core commit which made this more likely (although looking at the code, I've got to believe it would have been a problem in at least some scenarios anyway).

mikeryan’s picture

mikeryan
he/him
Primary language English
Location Murphysboro, IL, USA
CreditAttribution: mikeryan at Virtuoso Performance commented

Ah, migrate_example worked perfectly with core 8.1.1, but broke with my 8.1.x pulled May 12 - I've never done git bisect before, looks like a great opportunity to learn.

mikeryan’s picture

mikeryan
he/him
Primary language English
Location Murphysboro, IL, USA
CreditAttribution: mikeryan at Virtuoso Performance commented

The breaking commit was http://cgit.drupalcode.org/drupal/commit/?id=7426879 (issue #2694391: Separate cache bin for migrations) - this seems to dovetail with webflo's report #2724027: Config Import should invalide cache in MigrationConfigEntityPluginManager, so let's see if his suggested fix there will address this...

mikeryan’s picture

mikeryan
he/him
Primary language English
Location Murphysboro, IL, USA
CreditAttribution: mikeryan at Virtuoso Performance commented
FileSize
you_have_requested_a-2729665-5.patch504 bytes

So, the root cause is that our plugin manager is using a different cache service (@cache.discovery) than the new one that the core plugin manager is using (@cache.discovery_migration). Modifying our service definition to match (the attached patch) works against current 8.1.x/8.2.x, but presents a versioning problem - it'll break against 8.1.1, which is the current stable release of Drupal. We could be more dynamic, I think, looking up the core plugin manager's cache and using that. Or, rethink the whole relationship of the plugin managers...

webflo’s picture

webflo CreditAttribution: webflo at UEBERBIT GmbH commented 
--- a/migrate_plus.services.yml
+++ b/migrate_plus.services.yml

@@ -7,4 +7,4 @@ services:
-    arguments: ['@module_handler', '@cache.discovery', '@language_manager']
+    arguments: ['@module_handler', '@cache.discovery_migration', '@language_manager']

We could inherit all parent services by using "parent: plugin.manager.migration". Should work in both versions.

webflo’s picture

webflo CreditAttribution: webflo at UEBERBIT GmbH commented

Or, rethink the whole relationship of the plugin managers...

I was wondering if we could add all config entity based migrations via a plugin deriver.

  • mikeryan committed fe6c18c on 8.x-2.x 
    Issue #2729665 by mikeryan, webflo: You have requested a non-existent...
mikeryan’s picture

mikeryan
he/him
Primary language English
Location Murphysboro, IL, USA
CreditAttribution: mikeryan at Virtuoso Performance commented
Status: Active » Fixed

Ah yes, parent works nicely for the immediate problem - thanks!

Let's pick up on possibly refactoring the manager in the other issue...

  • mikeryan committed da5aa9a on 8.x-2.x 
    Issue #2729665 by mikeryan, webflo: You have requested a non-existent...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

ssibal’s picture

ssibal CreditAttribution: ssibal commented
Status: Closed (fixed) » Active

This is still a serious issue. The two patches above is not solving anything for stable Drupal (8.1.1, 8.1.2).

  • 'cache.discovery_migration' I was not able to find in 8.1.x-dev, or anywhere within 8.1.1 or 8.1.2

Any solution / advice to solve this for stable versions?

mikeryan’s picture

mikeryan
he/him
Primary language English
Location Murphysboro, IL, USA
CreditAttribution: mikeryan at Virtuoso Performance commented
Status: Active » Closed (fixed)

migrate_plus currently works as best it can with 8.1.x and 8.2.x - there are still circumstances where the missing cckfield plugin manager rears it's ugly head, but that needs to be fixed in core: #2560795: Source plugins have a hidden dependency on migrate_drupal. The best way forward is to get that core fix in as soon as possible.

dman’s picture

dman CreditAttribution: dman as a volunteer commented

Not to re-open, but just to provide clues for folk who still get this error this week (2016-06-06) ... This it the SRP for the reported error.

Problem can be easily replicated by instantiating a new, clean D8, and then starting the drush (or for me, for a change, drupal/console)
drupal migrate:setup
and pointing it at a D6 DB.
Give it the required old site credentials and then
FAIL.

With the above error.
Call to a member function createInstancesByTag() on a non-object in {...}/vendor/drupal/console/src/Command/Shared/MigrationTrait.php

It may help to :
enable additional migration helper modules on the target site before proceeding. Because that seems to be the overall issue.

If using drupal/console (with the appropriate target site context working):
drupal module:install migrate_drupal

Trying
drupal migrate:setup
again after that seems to help you progress.

ressa’s picture

ressa CreditAttribution: ressa as a volunteer commented

This happens to me if I issue the command drush migrate-fields-source some_migration, but not in any other situations, so it's not really a deal breaker. But the command sometimes go through, I just can't figure out why it works, when it actually does show the field sources :-)