We use the Feeds module as an interface between an inventory management system and the website.
Within the inventory management system, automotive spare parts are entered, modified, deleted, etc.. Every single save in the system generates a new CSV file, which is saved to a specific directory on the website's web server.
The Feeds module then retrieves each file from there using a cron job. This cron job runs every 60 seconds.

This worked very well with Drupal 7.
Back then, we used the "Feeds directory fetcher" module (https://www.drupal.org/project/feeds_fetcher_directory).
It ensured that each file was moved to a different directory after being imported.

Unfortunately, this module no longer works with Drupal 8 and later versions.
But we did conduct initial—and seemingly successful—tests using the current directory fetcher built into Feeds.
However, we have now encountered a problem that must be resolved before we can actually use the upgrade:

If the files remain in the same directory, they are repeatedly re-imported.

We discovered this after changing the status of a few spare parts from "published" to "unpublished" and back again.
This is reproducible even with just a single spare part (= always the same node ID!):

1. We place a file in the directory: 1.csv
→ Value for "Status" column: 0
→ Cron job triggers the feed → the spare part is set to "Unpublished" (= correct)

2. The next cron run without a new file for this node shows "no new posts" (= correct)

3. We place the file 2.csv in the directory (for the same node ID)
→ Value for "Status" column: 1
→ Cron job → the spare part is correctly set to "Published" (= correct)

4. As early as the next cron runs, both files are imported again (= incorrect!)

5. For every additional csv-file, all previous files are also imported (= incorrect!)

What can we do?

Thanks!

CommentFileSizeAuthor
2.csv93 bytes4kant
1.csv93 bytes4kant

Comments

4kant created an issue.

joelpittet’s picture

From the sounds of it, it might be worth for you to offer to maintain or co-maintain the module that you mentioned. Reach out to Steve Jones, see if you can revive the features.

It sounds niche and there are plenty of features in feeds. Feel free to ask questions on slack if you have trouble integrating but the hooks and events and plugins in D8+ should be enough to get you the module up and running.