Problem/Motivation

When trying to import values for entity reference fields it is possible that the referenced entity does not exist yet and that the entity in question either appears later in the feed or that it gets imported via a different feed.

A good example where this could happen is when importing authors and articles. These need to get imported separately, using two feed entities.

See for example the following CSV files:

Authors

name email
Morticia morticia@example.com
Fester fester@example.com

This CSV file represents users to import. 'name' is set an unique target.

Articles

title author
Lorem ipsum Morticia
Ut wisi enim ad minim veniam Morticia
Nam liber tempor Fester

This CSV file represents nodes to import. 'title' is set an unique target. 'author' is mapped to a field that is referencing an user.

It is possible that articles get imported before the authors are imported. In this case the referenced author does not exist yet. But after the import of authors, when importing articles again, the articles are not updated. The references remain empty. This is because Feeds did not detect a change in the source and therefore does not update.

An other example, where entities are referenced within the same file:

Terms

name parent
Lorem ipsum
Nam liber tempor Lorem ipsum
Mirum est notare Eodem modo typi
Eodem modo typi

This CSV file represents terms to import. 'name' is mapped to the term's name, 'parent' is mapped to a field that is references an other term.

In the above example, for 'Nam liber tempor' - that references 'Lorem ipsum' - the entity reference will be imported with success on the first import. This is because 'Lorem ipsum' appears earlier in the file and thus is imported first. But this does not happen for 'Mirum est notare', because 'Eodem modo typi' appears later in the file and therefore does not exist at the time 'Mirum est notare' is imported first. On a subsequent import, the reference for this term will remain empty because the row for 'Mirum est notare' did not change.

Proposed resolution

When a referenced entity is not found, reset the feeds item's hash value. This will instruct Feeds that it needs to update the item on the next import.

Comments

MegaChriz created an issue. See original summary.

megachriz’s picture

Status: Active » Needs review
StatusFileSize
new13.14 KB

Work in progress patch. I want to add a test that tests importing terms and their parent as in the second example from the issue summary.

megachriz’s picture

StatusFileSize
new20.11 KB
new8.86 KB

And here is a test with a file where referenced items can appear later in the file. It also fixes a bug introduced in the previous patch: in some cases when passing an empty value to the entity reference target, the hash of the feed item was reset as well. This bug was catched with the new test: for the term 'Europe' there is no parent defined in the CSV file, thus an empty value for the entity reference target.

megachriz’s picture

Self-review:

  1. +++ b/src/Feeds/Target/EntityReference.php
    @@ -102,6 +104,61 @@ class EntityReference extends FieldTargetBase implements ConfigurableTargetInter
    +    // Remove empty values first.
    +    $raw_values = array_filter($raw_values);
    +    foreach ($raw_values as $key => $value) {
    +      if (isset($value['target_id']) && strlen(trim($value['target_id'])) === 0) {
    +        unset($raw_values[$key]);
    +      }
    +    }
    +
    +    if (empty($raw_values)) {
    +      // No values given. Let the parent handle these.
    +      return parent::setTarget($feed, $entity, $field_name, $raw_values);
    +    }
    ...
    +    $values = $this->prepareValues($raw_values);
    +    if (count($values) !== count($raw_values)) {
    +      // Some of the given values were not found.
    +      $reimport = TRUE;
    +    }
    

    This looks a bit too complex. Filtering out empty values first implies that setTarget() exactly knows what prepareValues() does. If the logic of prepareValues() changes, this code could introduce a bug.

  2. +++ b/src/Feeds/Target/EntityReference.php
    @@ -102,6 +104,61 @@ class EntityReference extends FieldTargetBase implements ConfigurableTargetInter
    +      foreach ($values as $subvalues) {
    +        if (array_key_exists('target_id', $subvalues) && is_null($subvalues['target_id'])) {
    +          $reimport = TRUE;
    +          break;
    +        }
    +      }
    

    This loop is not necessary if $reimport is already true.

  3. +++ b/tests/src/Kernel/Feeds/Target/EntityReferenceTest.php
    @@ -0,0 +1,421 @@
    +   * When importing a feed with that references items that are imported by an
    

    "with that"

  4. +++ b/tests/src/Kernel/Feeds/Target/EntityReferenceTest.php
    @@ -0,0 +1,421 @@
    +    // Create two feed types.
    

    Explain why two feed types need to be created.

  5. +++ b/tests/src/Kernel/Feeds/Target/EntityReferenceTest.php
    @@ -0,0 +1,421 @@
    +   * Tests if articles get an assigned author later.
    

    "assigned" can be omitted.

  6. +++ b/tests/src/Kernel/Feeds/Target/EntityReferenceTest.php
    @@ -0,0 +1,421 @@
    +    // Assert two created nodes.
    +    $this->assertNodeCount(3);
    

    'two' does not match '3'.

  7. +++ b/tests/src/Kernel/Feeds/Target/EntityReferenceTest.php
    @@ -0,0 +1,421 @@
    +    // Assert that the node doesn't currently have an author.
    

    the first node.

  8. +++ b/tests/src/Kernel/Feeds/Target/EntityReferenceTest.php
    @@ -0,0 +1,421 @@
    +    // And re-import first feed.
    

    "And re-import first feed. Previously imported articles now should get an author."

  9. +++ b/tests/src/Kernel/Feeds/Target/EntityReferenceTest.php
    @@ -0,0 +1,421 @@
    +    // Reload node.
    

    "Reload node 1 and check if it got an author."

  10. +++ b/tests/src/Kernel/Feeds/Target/EntityReferenceTest.php
    @@ -0,0 +1,421 @@
    +    // And re-import first feed again.
    

    "And re-import first feed again. No nodes should get updated."

  11. +++ b/tests/src/Kernel/Feeds/Target/EntityReferenceTest.php
    @@ -0,0 +1,421 @@
    +    // Assert that 'Belgium' did not get a parent assigned, but the Netherlands
    

    Put "Netherlands" in quotes.

megachriz’s picture

Title: Optionally reset hash when an entity reference wasn't found » Reset hash when an entity reference wasn't found
StatusFileSize
new21.49 KB
new9.59 KB

This is a simpler implementation that introduces the exception class "ReferenceNotFoundException". Hopefully this also works.
Also expanded the docs in the test to make things a bit more clear as noted in #4.

Status: Needs review » Needs work

The last submitted patch, 5: feeds-entityreference-hash-reset-2989789-5.patch, failed testing. View results

megachriz’s picture

Status: Needs work » Needs review
StatusFileSize
new23.26 KB
new3.14 KB

Fixing unit test EntityReferenceTest. Also make ReferenceNotFoundException extend EmptyFeedException for backwards compatibility. EntityReference::setTarget() is catching ReferenceNotFoundException first, so we should be good.

  • MegaChriz committed f90baf3 on 8.x-3.x
    Issue #2989789 by MegaChriz: Reset hash when an entity reference wasn't...
megachriz’s picture

Status: Needs review » Fixed

Committed #7.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.