Problem/Motivation

I have been developing a site that uses migrate fairly heavily to migrate data from a D6 site to a D8 site. Migration has run just fine on the site until doing an upgrade to 8.5.4 - the migration runs in 8.5.3, but after upgrading to either 8.5.4 or 8.5.5 migration consistently is broken. The symptom, for example when running drush ms is

Error: Call to a member function getSetting() on null in             [error]
Drupal\migrate_drupal\Plugin\migrate\EntityReferenceTranslationDeriver->getDerivativeDefinitions()
(line 101 of
/home/elections/domains/develections.cruiskeenconsulting.com/public_html/core/modules/migrate_drupal/src/Plugin/migrate/EntityReferenceTranslationDeriver.php)
#0
/home/elections/domains/develections.cruiskeenconsulting.com/public_html/core/lib/Drupal/Component/Plugin/Discovery/DerivativeDiscoveryDecorator.php(101):
Drupal\migrate_drupal\Plugin\migrate\EntityReferenceTranslationDeriver->getDerivativeDefinitions(Array)
#1
/home/elections/domains/develections.cruiskeenconsulting.com/public_html/core/lib/Drupal/Component/Plugin/Discovery/DerivativeDiscoveryDecorator.php(87):
Drupal\Component\Plugin\Discovery\DerivativeDiscoveryDecorator->getDerivatives(Array)
#2
/home/elections/domains/develections.cruiskeenconsulting.com/public_html/core/modules/migrate/src/Plugin/MigrationPluginManager.php(256):
Drupal\Component\Plugin\Discovery\DerivativeDiscoveryDecorator->getDefinitions()
#3
/home/elections/domains/develections.cruiskeenconsulting.com/public_html/core/lib/Drupal/Core/Plugin/DefaultPluginManager.php(175):
Drupal\migrate\Plugin\MigrationPluginManager->findDefinitions()
#4
/home/elections/domains/develections.cruiskeenconsulting.com/public_html/core/modules/migrate/src/Plugin/MigrationPluginManager.php(109):
Drupal\Core\Plugin\DefaultPluginManager->getDefinitions()
#5
/home/elections/domains/develections.cruiskeenconsulting.com/public_html/modules/contrib/migrate_tools/migrate_tools.drush.inc(499):
Drupal\migrate\Plugin\MigrationPluginManager->createInstances(Array)
#6
/home/elections/domains/develections.cruiskeenconsulting.com/public_html/modules/contrib/migrate_tools/migrate_tools.drush.inc(149):
drush_migrate_tools_migration_list('')
#7 phar:///usr/bin/drush/includes/command.inc(422):
drush_migrate_tools_migrate_status()
#8 phar:///usr/bin/drush/includes/command.inc(231):
_drush_invoke_hooks(Array, Array)
#9 phar:///usr/bin/drush/includes/command.inc(199): drush_command()
#10 phar:///usr/bin/drush/lib/Drush/Boot/BaseBoot.php(67):
drush_dispatch(Array)
#11 phar:///usr/bin/drush/includes/preflight.inc(66):
Drush\Boot\BaseBoot->bootstrap_and_dispatch()
#12 phar:///usr/bin/drush/includes/startup.inc(462): drush_main()
#13 phar:///usr/bin/drush/includes/startup.inc(369):
drush_run_main(false, '/', 'Phar detected. ...')
#14 phar:///usr/bin/drush/drush(114): drush_startup(Array)
#15 /usr/bin/drush(10): require('phar:///usr/bin...')
#16 {main}.
Error: Call to a member function getSetting() on null in /home/elections/domains/develections.cruiskeenconsulting.com/public_html/core/modules/migrate_drupal/src/Plugin/migrate/EntityReferenceTranslationDeriver.php on line 101 #0 /home/elections/domains/develections.cruiskeenconsulting.com/public_html/core/lib/Drupal/Component/Plugin/Discovery/DerivativeDiscoveryDecorator.php(101): Drupal\migrate_drupal\Plugin\migrate\EntityReferenceTranslationDeriver->getDerivativeDefinitions(Array)
#1 /home/elections/domains/develections.cruiskeenconsulting.com/public_html/core/lib/Drupal/Component/Plugin/Discovery/DerivativeDiscoveryDecorator.php(87): Drupal\Component\Plugin\Discovery\DerivativeDiscoveryDecorator->getDerivatives(Array)
#2 /home/elections/domains/develections.cruiskeenconsulting.com/public_html/core/modules/migrate/src/Plugin/MigrationPluginManager.php(256): Drupal\Component\Plugin\Discovery\DerivativeDiscoveryDecorator->getDefinitions()
#3 /home/elections/domains/develections.cruiskeenconsulting.com/public_html/core/lib/Drupal/Core/Plugin/DefaultPluginManager.php(175): Drupal\migrate\Plugin\MigrationPluginManager->findDefinitions()
#4 /home/elections/domains/develections.cruiskeenconsulting.com/public_html/core/modules/migrate/src/Plugin/MigrationPluginManager.php(109): Drupal\Core\Plugin\DefaultPluginManager->getDefinitions()
#5 /home/elections/domains/develections.cruiskeenconsulting.com/public_html/modules/contrib/migrate_tools/migrate_tools.drush.inc(499): Drupal\migrate\Plugin\MigrationPluginManager->createInstances(Array)
#6 /home/elections/domains/develections.cruiskeenconsulting.com/public_html/modules/contrib/migrate_tools/migrate_tools.drush.inc(149): drush_migrate_tools_migration_list('')
#7 phar:///usr/bin/drush/includes/command.inc(422): drush_migrate_tools_migrate_status()
#8 phar:///usr/bin/drush/includes/command.inc(231): _drush_invoke_hooks(Array, Array)
#9 phar:///usr/bin/drush/includes/command.inc(199): drush_command()
#10 phar:///usr/bin/drush/lib/Drush/Boot/BaseBoot.php(67): drush_dispatch(Array)
#11 phar:///usr/bin/drush/includes/preflight.inc(66): Drush\Boot\BaseBoot->bootstrap_and_dispatch()
#12 phar:///usr/bin/drush/includes/startup.inc(462): drush_main()
#13 phar:///usr/bin/drush/includes/startup.inc(369): drush_run_main(false, '/', 'Phar detected. ...')
#14 phar:///usr/bin/drush/drush(114): drush_startup(Array)
#15 /usr/bin/drush(10): require('phar:///usr/bin...')
#16 {main}
Error: Call to a member function getSetting() on null in Drupal\migrate_drupal\Plugin\migrate\EntityReferenceTranslationDeriver->getDerivativeDefinitions() (line 101 of /home/elections/domains/develections.cruiskeenconsulting.com/public_html/core/modules/migrate_drupal/src/Plugin/migrate/EntityReferenceTranslationDeriver.php).
Drush command terminated abnormally due to an unrecoverable error.   [error]

I'm not quite certain how to proceed at this point, this sounds quite a lot like it may be related to https://www.drupal.org/project/drupal/issues/2912348 - but that appears to be fixed. If there is any help possible here I'd appreciate it.

Steps to reproduce

TBD

Proposed resolution

Add a check for NULL before using the returned array. The docs for \Drupal\Core\Entity\EntityFieldManagerInterface::getFieldDefinitions() doesn't state that NULL can be returned.

Remaining tasks

Manual testing - There are 3 reports of successful manual testing of the patch in #3, See #8, #10, #12, and #26.
Write a test - To do this a field needs to be created that does not have a definition. How to do that?
Review
Commit

CommentFileSizeAuthor
#23 2984460-3.patch1 KBquietone
#14 interdiff_3-14.txt13.93 KBneelam_wadhwani
#14 2984460-14.patch1.59 KBneelam_wadhwani
#3 2984460-3.patch1 KBmaxocub

Comments

Steve Hanson created an issue. See original summary.

jumoke’s picture

jumoke commented

Hi Steve, did you ever figure this out? I am running into the same issue with 8.5, wondering what your fix was.

maxocub’s picture

maxocub commented
Category: Support request » Bug report
Issue tags: +Needs tests
StatusFileSize
new 2984460-3.patch1 KB

Can you try this patch? I think it should fix it.

maxocub’s picture

maxocub commented
Status: Active » Needs work

NW for the tests.

steve hanson’s picture

steve hanson commented

Yes - patch in comment 3 did the trick - migration is working fine again. Thank you from the bottom of my heart.

jumoke’s picture

jumoke commented

Awesome! Patch in #3 works. Thank you so much Max

heddn’s picture

heddn
Primary language English
Location Nicaragua
commented
Title: Migration for my site broken from 8.5.4 on » Call to a member function getSetting() on null in Drupal\migrate_drupal\Plugin\migrate\EntityReferenceTranslationDeriver->getDerivativeDefinitions()
Version: 8.5.4 » 8.7.x-dev

Let's re-title and move this into 8.7's queue.

wgallop99’s picture

wgallop99 commented

Worked for me as well!!

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

ramonma1989’s picture

ramonma1989
Primary language Spanish
commented

#3 works for me. Thanks.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

vgardner’s picture

vgardner
Location London
commented

#3 worked for me. Cheers

neelam_wadhwani’s picture

neelam_wadhwani commented
Assigned: Unassigned » neelam_wadhwani
neelam_wadhwani’s picture

neelam_wadhwani commented
StatusFileSize
new 2984460-14.patch1.59 KB
new interdiff_3-14.txt13.93 KB

Did the changes and tested for drupal 8.9.x.

neelam_wadhwani’s picture

neelam_wadhwani commented
Assigned: neelam_wadhwani » Unassigned
Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 14: 2984460-14.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

prabha1997’s picture

prabha1997 commented
Assigned: Unassigned » prabha1997

I am working in this issue

dhirendra.mishra’s picture

dhirendra.mishra commented
Status: Needs work » Reviewed & tested by the community

#3 patch cleanely applies to 8.9.x.

This is then RTBC for me.

jungle’s picture

jungle
He/Him
Location Chongqing, China
commented
Status: Reviewed & tested by the community » Needs work
  1. 1 test did not pass in #14
  2. /var/lib/drupalci/workspace/jenkins-drupal_patches-33258/source/core/modules/migrate_drupal/src/Plugin/migrate/EntityReferenceTranslationDeriver.php
  line 102	Expected 1 space after closing parenthesis; found 0

    Coding standards check must pass

See https://www.drupal.org/pift-ci-job/1582583

jungle’s picture

jungle
He/Him
Location Chongqing, China
commented
Status: Needs work » Needs review 
+++ b/composer.json
@@ -13,7 +13,8 @@
+        "wikimedia/composer-merge-plugin": "^1.4",
+        "drupal/address": "1.x-dev"

Re #14. oh, no, drupal/address": "1.x-dev" should not be added

#3 patch is the right one to review. Set back to Needs review.

jungle’s picture

jungle
He/Him
Location Chongqing, China
commented

Would be great to have test coverage. But not sure whether it's a must-have.

jungle’s picture

jungle
He/Him
Location Chongqing, China
commented
Assigned: prabha1997 » Unassigned

@prabha1997 claimed to work on this, but no progress for 4 days. if you are still working on it, please assign to yourself again. Thanks!

quietone’s picture

quietone commented
StatusFileSize
new 2984460-3.patch1 KB

Had a go at writing a test for this with no progress. I don't see how to make a field without a definition. I'd like to know what was in the original db that caused this. If you see how to write the test, pleas comment.

Anyway, there are 3 reports of successful manual testing, #8, #10, #12.

The change itself is just to check for NULL before using the returned array. The docs for \Drupal\Core\Entity\EntityFieldManagerInterface::getFieldDefinitions() doesn't state that NULL can be returned.

I am re-uploading the successful patch from #3.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

mikelutz’s picture

mikelutz
He/Him
Primary language English
Location Michigan, USA
commented
Status: Needs review » Needs work

Setting this back to NW for tests again. If we can't duplicate the behavior in a test, then there may be some other issue here. If there is a bug in \Drupal\Core\Entity\EntityFieldManagerInterface::getFieldDefinitions() that it is returning NULL (or if the docs are just wrong) then we should address those.

alexrayu’s picture

alexrayu commented

#23 resolved the issue for me. Thanks!

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

quietone’s picture

quietone commented
Issue summary: View changes
Issue tags: +migrate-d6-d8

Update the IS. The original report was for a D6 source, so adding tag.

If you are using the patch please leave a comment stating the Drupal version of the source and destination sites. Thanks.

quietone credited bachbach.

quietone’s picture

quietone commented

Closed #3032739: Error: Call to a member function getSetting() on null as a duplicate, adding credit.

quietone’s picture

quietone commented

I can get this failure locally by running d7_node_type, d7_field, d7_field_instance and d7_node_complete with the taxonomy module disabled. So, I started writing a test but that I can't get to the execution of d7_node_complete. I get errors in d7_field_instance resulting in the fatal error in this issue #3192893: [META] Serialization issues in Migration tests. And then when I apply that I get

PHP Fatal error:  Uncaught Drupal\Core\Database\ConnectionNotDefinedException: The specified 
database connection is not defined: migrate in /var/www/html/core/lib/Drupal/Core/Database/Da
tabase.php:371
Stack trace:

Which is totally unexpected but might be a clue to #3123950: 'Field discovery failed for Drupal core version 7' errors if migration source database key is not 'migrate'. It is too late to investigate further. It will have to wait for another day.

thomaswalther’s picture

thomaswalther commented

I also got the error with drupal 8.9.13 and slick (8.x-2.2) in view with slick_views (8.x-2.3):

Nachricht	Error: Call to a member function getSetting() on null in Drupal\slick\SlickManager->prepareSettings() (Zeile 242 in /webroot/modules/slick/src/SlickManager.php)
#0 /webroot/modules/slick/src/SlickManager.php(310): Drupal\slick\SlickManager->prepareSettings()

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

4kant’s picture

4kant commented

Patch from #23 worked for me
Migration from 7.80 to 9.1.10

Thanks

vlad.dancer’s picture

vlad.dancer
Primary language Ukrainian
Location Kyiv
commented

I confirm too that patch from #23 works for me too.
Doing migration from 7.77 to 9.2.0

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

felubra’s picture

felubra
he/him
Primary language Portuguese, Brazil
Location Vitória, ES, Brazil
commented

The patch on #23 also worked for me in Drupal 9.4.0

grevil’s picture

grevil commented

We just ran into the same issue! Patch from #23 works perfect!

We are migrating a large site from Drupal 7 to Drupal 9.4.5. Re #31 our migrate key is ['migrate']['default'] so this doesn't seem to be the reason for us.

huzooka’s picture

huzooka
Primary language Hungarian
Location Hungary 🇭🇺🇪🇺
commented

You all are using Drush migrate commands probably. I think the issue is about Drush’s migrate command.

That command instantiates all migration plugin instances at a very early phase, and also performs a requirement check The problem with this approach is not just that it can eat your memory (because after this, the plugin instances stored in memory will also contain fully instantiated source and destination plugin instances which often store their host object in a property), but this will make your source/destination plugin instances contain entity* managers with potentially outdated static definition caches.

If you try to execute these follow up migrations after everything is finished, in a "new" PHP thread, the issue won’t happen.

Keep in mind that there is a fundamental difference between Migrate Drupal UI and Drush migrate: while the built-in UI executes each migration in a separate batch task (which also means a new PHP thread), Drush migrate import executes every migration in the same PHP thread.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

anybody’s picture

anybody
Primary language German
Location Porta Westfalica
commented

Thanks for the heads-up @huzooka do you see any chance to fix that? At least the memory-thing? Using the UI for large migrations sounds untypical.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

grevil’s picture

grevil commented

FYI, Patch #23 still applies cleanly on latest 11.x.

quietone’s picture

quietone commented
Status: Needs work » Postponed

The Migrate Drupal Module was approved for removal in #3371229: [Policy] Migrate Drupal and Migrate Drupal UI after Drupal 7 EOL.

This is Postponed. The status is set according to two policies. The Remove a core extension and move it to a contributed project and the Extensions approved for removal policies.

The deprecation work is in #3522602: [meta] Tasks to remove Migrate Drupal module and the removal work in #3522602: [meta] Tasks to remove Migrate Drupal module.

Migrate Drupal will not be moved to a contributed project. It will be removed from core after the Drupal 12.x branch is open.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.