Problem/Motivation
When trying to import values for entity reference fields it is possible that the referenced entity does not exist yet and that the entity in question either appears later in the feed or that it gets imported via a different feed.
A good example where this could happen is when importing authors and articles. These need to get imported separately, using two feed entities.
See for example the following CSV files:
Authors
| name | |
|---|---|
| Morticia | morticia@example.com |
| Fester | fester@example.com |
This CSV file represents users to import. 'name' is set an unique target.
Articles
| title | author |
|---|---|
| Lorem ipsum | Morticia |
| Ut wisi enim ad minim veniam | Morticia |
| Nam liber tempor | Fester |
This CSV file represents nodes to import. 'title' is set an unique target. 'author' is mapped to a field that is referencing an user.
It is possible that articles get imported before the authors are imported. In this case the referenced author does not exist yet. But after the import of authors, when importing articles again, the articles are not updated. The references remain empty. This is because Feeds did not detect a change in the source and therefore does not update.
An other example, where entities are referenced within the same file:
Terms
| name | parent |
|---|---|
| Lorem ipsum | |
| Nam liber tempor | Lorem ipsum |
| Mirum est notare | Eodem modo typi |
| Eodem modo typi |
This CSV file represents terms to import. 'name' is mapped to the term's name, 'parent' is mapped to a field that is references an other term.
In the above example, for 'Nam liber tempor' - that references 'Lorem ipsum' - the entity reference will be imported with success on the first import. This is because 'Lorem ipsum' appears earlier in the file and thus is imported first. But this does not happen for 'Mirum est notare', because 'Eodem modo typi' appears later in the file and therefore does not exist at the time 'Mirum est notare' is imported first. On a subsequent import, the reference for this term will remain empty because the row for 'Mirum est notare' did not change.
Proposed resolution
When a referenced entity is not found, reset the feeds item's hash value. This will instruct Feeds that it needs to update the item on the next import.
| Comment | File | Size | Author |
|---|---|---|---|
| #7 | interdiff-2989789-5-7.txt | 3.14 KB | megachriz |
| #7 | feeds-entityreference-hash-reset-2989789-7.patch | 23.26 KB | megachriz |
| #5 | interdiff-2989789-3-5.txt | 9.59 KB | megachriz |
| #5 | feeds-entityreference-hash-reset-2989789-5.patch | 21.49 KB | megachriz |
| #3 | interdiff-2989789-2-3.txt | 8.86 KB | megachriz |
Comments
Comment #2
megachrizWork in progress patch. I want to add a test that tests importing terms and their parent as in the second example from the issue summary.
Comment #3
megachrizAnd here is a test with a file where referenced items can appear later in the file. It also fixes a bug introduced in the previous patch: in some cases when passing an empty value to the entity reference target, the hash of the feed item was reset as well. This bug was catched with the new test: for the term 'Europe' there is no parent defined in the CSV file, thus an empty value for the entity reference target.
Comment #4
megachrizSelf-review:
This looks a bit too complex. Filtering out empty values first implies that setTarget() exactly knows what prepareValues() does. If the logic of prepareValues() changes, this code could introduce a bug.
This loop is not necessary if $reimport is already true.
"with that"
Explain why two feed types need to be created.
"assigned" can be omitted.
'two' does not match '3'.
the first node.
"And re-import first feed. Previously imported articles now should get an author."
"Reload node 1 and check if it got an author."
"And re-import first feed again. No nodes should get updated."
Put "Netherlands" in quotes.
Comment #5
megachrizThis is a simpler implementation that introduces the exception class "ReferenceNotFoundException". Hopefully this also works.
Also expanded the docs in the test to make things a bit more clear as noted in #4.
Comment #7
megachrizFixing unit test EntityReferenceTest. Also make ReferenceNotFoundException extend EmptyFeedException for backwards compatibility.
EntityReference::setTarget()is catching ReferenceNotFoundException first, so we should be good.Comment #9
megachrizCommitted #7.