Problem/Motivation

The data from a FeedsSource plugin is currently fetched during the mapping stage, in EntityProcessorBase::map(). This has two limitations:

  • Data from extra sources cannot be altered with Feeds Tamper (nor with a custom module using the parse event).
  • When a value in an extra source changes, but the base source stays the same, entities are not updated because Feeds only looks for changes on the base source.

Proposed resolution

Fetch the values from FeedsSource plugins earlier in the process, so their values can be altered with Feeds Tamper.

CommentFileSizeAuthor
#9 interdiff-2989743-7-9.txt709 bytesmegachriz
#9 feeds-source-plugins-fetch-earlier-2989743-9.patch13.44 KBmegachriz
#7 interdiff-2989743-5-7.txt940 bytesmegachriz
#7 feeds-source-plugins-fetch-earlier-2989743-7.patch13.43 KBmegachriz
#5 interdiff-2989743-3-5.txt789 bytesmegachriz
#5 feeds-source-plugins-fetch-earlier-2989743-5.patch13.62 KBmegachriz
#3 feeds-source-plugins-fetch-earlier-2989743-3.patch12.85 KBmegachriz
#2 feeds-source-plugins-fetch-earlier-2989743-2-tests-only.patch9.79 KBmegachriz

Comments

MegaChriz created an issue. See original summary.

megachriz’s picture

megachriz
Primary language Dutch
commented
Status: Active » Needs work
StatusFileSize
new feeds-source-plugins-fetch-earlier-2989743-2-tests-only.patch9.79 KB

Attached a test that demonstrates the issue.

megachriz’s picture

megachriz
Primary language Dutch
commented
Status: Needs work » Needs review
StatusFileSize
new feeds-source-plugins-fetch-earlier-2989743-3.patch12.85 KB

This should do the trick. Let's see if it has unwanted side effects.

Status: Needs review » Needs work

The last submitted patch, 3: feeds-source-plugins-fetch-earlier-2989743-3.patch, failed testing. View results

megachriz’s picture

megachriz
Primary language Dutch
commented
Status: Needs work » Needs review
StatusFileSize
new feeds-source-plugins-fetch-earlier-2989743-5.patch13.62 KB
new interdiff-2989743-3-5.txt789 bytes

Updated Drupal\Tests\feeds\Unit\EventSubscriber\LazySubscriberTest.

megachriz’s picture

megachriz
Primary language Dutch
commented
Status: Needs review » Reviewed & tested by the community

Looked through the code again, looks good to me.

megachriz’s picture

megachriz
Primary language Dutch
commented
Status: Reviewed & tested by the community » Needs review
StatusFileSize
new feeds-source-plugins-fetch-earlier-2989743-7.patch13.43 KB
new interdiff-2989743-5-7.txt940 bytes

With #2990155: Only update when mapped fields are updated being committed - which added FeedTypeInterface::getMappedSources() - I believe that LazySubscriber::getMappedSourcePlugins() could be made slightly simpler.

Status: Needs review » Needs work

The last submitted patch, 7: feeds-source-plugins-fetch-earlier-2989743-7.patch, failed testing. View results

megachriz’s picture

megachriz
Primary language Dutch
commented
Status: Needs work » Needs review
StatusFileSize
new feeds-source-plugins-fetch-earlier-2989743-9.patch13.44 KB
new interdiff-2989743-7-9.txt709 bytes

This hopefully fixes the failing test.

  • MegaChriz committed ce546a8 on 8.x-3.x 
    Issue #2989743 by MegaChriz: Fetch values from FeedsSource plugins...
megachriz’s picture

megachriz
Primary language Dutch
commented
Status: Needs review » Fixed

Committed #9.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.