I have the following use case: 2 feeds importer, both mapping to user entity, username used as unique.
The workflow of the application requires to import each time 2 different files ( first file with feed importer 1, second file with feed importer 2).
For both feeds skip hask check is off . Importing alternatively, every feed destroy the hash of the other( there is one table and one entity).
Even if a choose skip hash check for feeds 1, still destroy the hash for feeds2.
P.S. The solution of merging the files and import them into one feed is excluded by the end-user.

CommentFileSizeAuthor
#1 skip-hash-check-for_real-2305751-1.patch848 bytesDrupa1ish
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Drupa1ish’s picture

Drupa1ish CreditAttribution: Drupa1ish commented
Status: Active » Needs review
FileSize
skip-hash-check-for_real-2305751-1.patch848 bytes

The patch suggest skip hash check for real: if a feed has skip hash check enabled, during import we should keep the previously saved hashes.

Status: Needs review » Needs work

The last submitted patch, 1: skip-hash-check-for_real-2305751-1.patch, failed testing.

twistor’s picture

twistor CreditAttribution: twistor commented

Hmm... I don't know about this. Skipping the check doesn't imply that it won't update the hash value.

Why does it matter? If you have "skip hash check" on for both, then they should just always update all of them.

Drupa1ish’s picture

Drupa1ish CreditAttribution: Drupa1ish commented

I love this hash feature.
After the import, I need to do heavy operations on each user at hook_feeds_after_save. So is very relevant to perform only on the affected users by the import. Hash helps a lot.

So I need hash to be enabled for both feeds, in the first place.
Only because #2306305: Multiple feeds importer on the same entity should not override each other’s hashes is not possible at the moment ( I suppose it requires bigger architectural changes ) , i suggested the workaround in this patch.

Rather that not having hash for both feeds, I prefer to have it only for one feed and for the other to skip it.

Anyway, it seems natural that skip hash check won't update the hash value.

lmeurs’s picture

lmeurs CreditAttribution: lmeurs commented

I am not sure about the added functionality by the patch since I think the hash should always represent the most recent imported data.

@EuroDomenii: Can you explain why #2306305: Multiple feeds importer on the same entity should not override each other’s hashes is not possible at the moment? Though I have not tested the patch from that issue yet, the solution could solve problems with feeds_items importer ID's and hashes being overwritten when multiple importers update the same entities. Also see my comment at #1539224-63: Add support for unique fields to be unique site wide.

MegaChriz’s picture

MegaChriz CreditAttribution: MegaChriz commented

I agree with lmeurs and twistor. Skipping hash check does not imply that it won't update the hash value.

Let's illustrate this with an example. Say you have a feed that contains 10 000 items. Due to a major change in the source, 9 000 items get changed. For whatever reason the importer had the "Skip hash check" option enabled and all 10 000 items get updated on the site. Then you disable the option "Skip hash check". With the current behaviour, Feeds knows which items it updated (all of them) so when the next import runs (and the source hasn't changed since then) no items are updated. If, however, the hashes were not overwritten, Feeds would still have the hashes from before the major change in the source and in that case it would update 9 000 items on the site that were not changed!

I would say "won't fix" to this one, but I'll leave it open for now to hear why #2306305: Multiple feeds importer on the same entity should not override each other’s hashes would need this change.

Drupa1ish’s picture

Drupa1ish CreditAttribution: Drupa1ish commented

@MegaChriz I don't dispute your use case. Is is hypothetical or for real? It seems a little bit strange to enable and disable skip hash.
Anyway, 2 feeds importer on the same entity happened to me in a project and it seems pretty common.
Believe me, every feed destroy the hash of the other. Daily, one feed was running for 5 min, other for 1 hour. It was very inconvenient.
The patch solved my problem( both feeds 10 min).
Maybe a solution is to introduce a variable to switch to both behaviors, depending on user needs.

Drupa1ish’s picture

Drupa1ish CreditAttribution: Drupa1ish commented

@lmeurs the comment "is not possible" is obsolete, from July 19, 2014, and the patch #2306305-2: Multiple feeds importer on the same entity should not override each other’s hashes is from November 27, 2014

lmeurs’s picture

lmeurs CreditAttribution: lmeurs commented

@Megachriz: At first I too thought comment #2306305-2: Multiple feeds importer on the same entity should not override each other’s hashes stated that the patch from #2306305 needed the patch from #2305751, but I guess EuroDomenii meant the patch from #2306305 was based on a patched version of the module.

@EuroDomenii: Thank you for your quick response, great to hear the solution from #2306305 still is viable. If #2306305 solves your issues and ours as well, shall we continue the discussion over there and close this one?

Drupa1ish’s picture

Drupa1ish CreditAttribution: Drupa1ish commented

Due to an outstanding deadline, I cannot re-dive into this issue for the moment. I am really sorry.
Of course, I shall follow the comments to see where this leads to.

Drupa1ish’s picture

Drupa1ish CreditAttribution: Drupa1ish commented

Do no apply this patch directly, because is included in patch #2306305-6: Multiple feeds importer on the same entity should not override each other’s hashes