Problem/Motivation

When a FeedsTarget plugin cannot be found for some reason, an import crashes with the following error:

Call to a member function getPluginId() on null in Drupal\feeds\Entity\FeedType->getTargetPlugin()

The mapping page also crashes, with the following error:

Error: Call to a member function hasProperty() on null in Drupal\feeds\Form\MappingForm->buildRow()

Steps to reproduce

Method 1:

  1. Create a feed type and configure mappings.
  2. Export the config.
  3. In the exported config, change one of the target names into a non-existent target.
    Example, change title into non_existent:
    mappings:
      -
        target: title
        map:
          value: title
    
    mappings:
      -
        target: non_existent
        map:
          value: title
    
  4. Import the config.
  5. Go to the mapping page.

Method 2:

  1. Create a feed type and configure mappings.
  2. Remove one of the used FeedsTarget plugins.
  3. Go to the mapping page.

Proposed resolution

In FeedType::getTargetPlugin(), throw an exception when a target plugin is not found. The exception message should say which target is not found.
Additionally, to prevent the mapping page from crashing, the mapping form should catch that exception.

A functional test is needed that would do the following:

  1. Add a feed type with a fake target.
  2. Assert that the mapping page still loads correctly.
  3. Assert that an import fails gracefully.
  4. Ensure the mapping for the fake target can be deleted on the mapping page.

Remaining tasks

  1. Throw an exception in FeedType::getTargetPlugin()
  2. Catch this exception in MappingForm.
  3. Add a functional test (see proposed resolution for details).

User interface changes

The mapping page and the import no longer crashes when a FeedsTarget plugin is not found.

API changes

None.

Data model changes

None.

Original report by turpentyne

I'm doing a couple CSV import test runs on a local basic install of Drupal 8.5.3, with php 7.1.12.

I get this error, if I create a new test import:

An AJAX HTTP error occurred.
HTTP Result Code: 200
Debugging information follows.
Path: /batch?id=39&op=do_nojs&op=do
StatusText: OK
ResponseText:

Steps that get me to the error:

I have a CSV file with 16 basic text fields, one having html.
I create a feed type. Settings are: Upload -> CSV --> Node --> testct --> as often as possible.
I map 12 of the fields in CSV file, one set to full html.
I go to content, click feeds and create a new import. Everything works. 200 entries added.
I saw an html character I didn't want - it rendered literal (& #44;). I replaced it with \, in the CSV.
I created a new feed import (just to have clarity of different attempts), and this, is when the error popped up. Import failure.
I deleted all the nodes that the previous import created and tried again. The error appeared again.
I went back to content/feeds and deleted all the feed imports there, and then created a new one. This time, it worked.

Comments

turpentyne created an issue. See original summary.

turpentyne’s picture

Issue summary: View changes
megachriz’s picture

Hi, thanks for filing a bug report with details!

The AJAX error is quite general (although it looks strange that the code is '200' which means that the request was successful).
Can you check the logs for more details about the error and report these here?

turpentyne’s picture

hah! Of course, now I'm having trouble regenerating the error. So I had to go back to when the error happened. I hope this is the right log:

Error: Call to a member function getPluginId() on null in Drupal\feeds\Entity\FeedType->getTargetPlugin() (line 400 of /Users/username/Sites/devdesktop/csmigrate6/modules/feeds/src/Entity/FeedType.php) #0 /Users/username/Sites/devdesktop/csmigrate6/modules/feeds/src/Feeds/Processor/EntityProcessorBase.php(812): Drupal\feeds\Entity\FeedType->getTargetPlugin(3) #1 /Users/username/Sites/devdesktop/csmigrate6/modules/feeds/src/Feeds/Processor/EntityProcessorBase.php(151): Drupal\feeds\Feeds\Processor\EntityProcessorBase->map(Object(Drupal\feeds\Entity\Feed), Object(Drupal\node\Entity\Node), Object(Drupal\feeds\Feeds\Item\DynamicItem)) #2 /Users/username/Sites/devdesktop/csmigrate6/modules/feeds/src/EventSubscriber/LazySubscriber.php(94): Drupal\feeds\Feeds\Processor\EntityProcessorBase->process(Object(Drupal\feeds\Entity\Feed), Object(Drupal\feeds\Feeds\Item\DynamicItem), Object(Drupal\feeds\State)) #3 [internal function]: Drupal\feeds\EventSubscriber\LazySubscriber->Drupal\feeds\EventSubscriber\{closure}(Object(Drupal\feeds\Event\ProcessEvent), 'feeds.process', Object(Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher)) #4 /Users/username/Sites/devdesktop/csmigrate6/core/lib/Drupal/Component/EventDispatcher/ContainerAwareEventDispatcher.php(111): call_user_func(Object(Closure), Object(Drupal\feeds\Event\ProcessEvent), 'feeds.process', Object(Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher)) #5 /Users/username/Sites/devdesktop/csmigrate6/modules/feeds/src/Event/EventDispatcherTrait.php(37): Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher->dispatch('feeds.process', Object(Drupal\feeds\Event\ProcessEvent)) #6 /Users/username/Sites/devdesktop/csmigrate6/modules/feeds/src/Plugin/QueueWorker/FeedRefresh.php(194): Drupal\feeds\Plugin\QueueWorker\FeedQueueWorkerBase->dispatchEvent('feeds.process', Object(Drupal\feeds\Event\ProcessEvent)) #7 /Users/username/Sites/devdesktop/csmigrate6/modules/feeds/src/Plugin/QueueWorker/FeedRefresh.php(93): Drupal\feeds\Plugin\QueueWorker\FeedRefresh->doProcess(Object(Drupal\feeds\Entity\Feed), Object(Drupal\feeds\Feeds\Item\DynamicItem)) #8 /Users/username/Sites/devdesktop/csmigrate6/core/lib/Drupal/Core/Cron.php(179): Drupal\feeds\Plugin\QueueWorker\FeedRefresh->processItem(Array) #9 /Users/username/Sites/devdesktop/csmigrate6/core/lib/Drupal/Core/Cron.php(144): Drupal\Core\Cron->processQueues() #10 /Users/username/Sites/devdesktop/csmigrate6/core/lib/Drupal/Core/ProxyClass/Cron.php(75): Drupal\Core\Cron->run() #11 /Users/username/Sites/devdesktop/csmigrate6/core/modules/automated_cron/src/EventSubscriber/AutomatedCron.php(65): Drupal\Core\ProxyClass\Cron->run() #12 [internal function]: Drupal\automated_cron\EventSubscriber\AutomatedCron->onTerminate(Object(Symfony\Component\HttpKernel\Event\PostResponseEvent), 'kernel.terminat...', Object(Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher)) #13 /Users/username/Sites/devdesktop/csmigrate6/core/lib/Drupal/Component/EventDispatcher/ContainerAwareEventDispatcher.php(111): call_user_func(Array, Object(Symfony\Component\HttpKernel\Event\PostResponseEvent), 'kernel.terminat...', Object(Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher)) #14 /Users/username/Sites/devdesktop/csmigrate6/vendor/symfony/http-kernel/HttpKernel.php(88): Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher->dispatch('kernel.terminat...', Object(Symfony\Component\HttpKernel\Event\PostResponseEvent)) #15 /Users/username/Sites/devdesktop/csmigrate6/vendor/stack/builder/src/Stack/StackedHttpKernel.php(32): Symfony\Component\HttpKernel\HttpKernel->terminate(Object(Symfony\Component\HttpFoundation\Request), Object(Drupal\Core\Render\HtmlResponse)) #16 /Users/username/Sites/devdesktop/csmigrate6/core/lib/Drupal/Core/DrupalKernel.php(642): Stack\StackedHttpKernel->terminate(Object(Symfony\Component\HttpFoundation\Request), Object(Drupal\Core\Render\HtmlResponse)) #17 /Users/username/Sites/devdesktop/csmigrate6/index.php(22): Drupal\Core\DrupalKernel->terminate(Object(Symfony\Component\HttpFoundation\Request), Object(Drupal\Core\Render\HtmlResponse)) #18 {main}.

megachriz’s picture

Call to a member function getPluginId() on null in Drupal\feeds\Entity\FeedType->getTargetPlugin() suggests that a configured target plugin could no longer be found. But that the error no longer comes back suggests that the target plugin that was missing is now found again. Unfortunately, the error doesn't tell us which target plugin was missing for a short time, so that makes it very hard to debug. It could perhaps have been an issue with caching. You are sure you didn't install/uninstall modules in between? Or moved around files from a module? Do you have or had modules installed that extend Feeds?

Either way, it would be good if the code checks that the target plugin exists and then throw a more specific error when it doesn't.

turpentyne’s picture

I bet you're right that it was a caching issue. The last plugin I removed was yesterday - Devel was causing errors with Feeds and several other import options. But nothing today.

Thank you!

megachriz’s picture

Title: ajax error 200 on re-import of same feed » Throw a more detailed error when a target plugin is not found in FeedType::getTargetPlugin()

Thanks for reporting back. Re-titling this issue.

What errors did Devel cause with Feeds? I remember I did encounter an error as well when trying to print the variables of a feed type with Kint. That could be the same issue.

turpentyne’s picture

Yeah, I think it is the same thing. But, if I remember right, It might've thrown the same errors even with Kint disabled.

I'd posted in the Support area about it, before realizing what it was:
https://www.drupal.org/forum/support/post-installation/2018-05-03/4-recu...

anybody’s picture

I agree with #5. That would make sense and help for better debugging. I've just run into "Error: Call to a member function getPluginId() on null". In my case a field had been deleted in the target entity type in the meantime.

The further problem is that the mapping can not be changed anymore due to a different error: admin/structure/feeds/manage/dguard_import_merchants/mapping so that the missing field can not be removed from the feeds mapping.

I solved that by adding the following lines in src/Form/MappingForm.php, Line 158:

      // Remove missing targets (=if the field does not exist anymore)
      if (empty($this->targets[$mapping['target']])) {
        $this->feedType->removeMapping($delta);
        return;
      }

which works and seems to be a practicable solution.

anybody’s picture

Status: Active » Needs review
StatusFileSize
new688 bytes

Patch solving the problem with deleted targets described in #9 attached. This doesn't add enhanced error messages for other problems, but in this case makes the mapping usable again and removed the deleted field.

megachriz’s picture

Hm, I don't like that the function is messing with the mapping. We can't be certain that the cause of not finding the target plugin is always gonna be missing a field. It could also be caused by a weird caching issue where somehow a target plugin is not being indexed. Or you when you are working on a target plugin and decide to rename it. Or when you accidentally moved around folders.

Note that the deleted field issue is most likely fixed in dev. With the latest dev version, when you save a feed type, dependencies on fields are calculated and saved on the feed type. This means that when you delete a field, you can see that it will affect a feed type and if you decide to continue deleting the field, Feeds automatically updates the mapping. See #2854914: Deleted Target Field Throws Fatal Error. Come to think of it, all feed types do need to be resaved after updating the Feeds module in order to save these dependency calculations on each feed type. I should be create a follow-up issue to resave all feed types in a database update.

I think throwing an exception is the right thing to do here. To avoid causing a fatal error on the mapping page, the mapping form should catch that exception. It would also be useful to add a functional test here. The test could programmatically add a fake target and then assert that the mapping page still loads correctly. Additionally, the test would try to delete the mapping for the fake target so we can also ensure that an user can clean up.

Would you like to work on that @Anybody? I'd like to make a new release of Feeds next week (or the week after) and I have a few things that are near ready that I like to have finished as well for that release. It would be nice if this thing could then make it too, but not sure if I'll have enough time to get to it.

anybody’s picture

Status: Needs review » Needs work

Hi MegaChriz,
I completely agree with #11. Currently I don't have the time to care for that, sorry. I guess you also don't. So it would be nice if someone may pick this up.

Thank you all so far!

paulgranted’s picture

I was still experiencing this issue:
MegaChriz's point about re-saving each item is spot on:
Come to think of it, all feed types do need to be resaved after updating the Feeds module in order to save these dependency calculations on each feed type. I should be create a follow-up issue to resave all feed types in a database update.

I had problems with a couple of feeds with the mappings of some fields being missing. Updating these and then resaving allowed the update to complete.

megachriz’s picture

@paulgranted
Resaving all feed types was in fact included as a database update since 8.x-3.0-alpha2.

megachriz’s picture

Issue summary: View changes
Issue tags: +sfdug2020, +Needs tests

Tagging issue for Global Contrib Weekend Event at SFDUG. Thursday Jan 23.

Also updated issue summary.

megachriz’s picture

Status: Needs work » Needs review
Issue tags: -Needs tests
StatusFileSize
new2.26 KB
new10.6 KB
new135.12 KB

I ran into this error while working on #2989279: Add a target to entity ID and then switching branches (on the Feeds git repo).

This patch prevents the fatal error in the following way:

  • It adds an exception class called "MissingTargetException".
  • It adds a placeholder target definition class called "MissingTargetDefinition".
  • The mapping form code checks if the target definition exists and if not, it creates a placeholder target definition of type MissingTargetDefinition.
  • CSS is added to highlight the mapping row that has got errors.

See the image for how this looks like in the UI:

anybody’s picture

Thank you @MegaChriz, I'll review that soon! :)

The last submitted patch, 16: feeds-missing-target-2969617-16-tests-only.patch, failed testing. View results

anybody’s picture

Status: Needs review » Reviewed & tested by the community

Sorry, finally I was able to review and test this. It works great! :) Wonderful work once again, you are wonderful.
RTBC!

  • MegaChriz committed a81c8b8 on 8.x-3.x
    Issue #2969617 by MegaChriz, Anybody: Throw a more detailed error when a...
megachriz’s picture

Status: Reviewed & tested by the community » Fixed

@Anybody
Thank you very much for your review and your positive words. :)

Committed #16

megachriz’s picture

Status: Fixed » Needs review
StatusFileSize
new2.71 KB

Hm, recent changes introduced new failures, apparently.

Patch to fix issues.

  • MegaChriz committed d7793dc on 8.x-3.x
    by MegaChriz: Fixed test failure caused by issue #2969617.
    
megachriz’s picture

Status: Needs review » Fixed

Committed #22.

anybody’s picture

Great work, thank you very very much @MegaChriz!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.