feeds 8.x-3.0-beta3

Big changes: New Feeds Log module, Drupal 10 support

New Feeds Log module!

A module is added to Feeds with which you can track your imports. It lets you see in detail which items were created or updated and which one failed.
See the video for a quick introduction:
https://youtu.be/P3ipggf5Irs

Other notable additions and changes

• Multiple feeds to update the same content

  Imported content can now be tracked by multiple feeds. This way Feeds can properly see if content by each feed has changed. Previously, when you used more than one feed to update the same content, Feeds would see all content updated by feed 1 as changed when importing these with feed 2. This caused that a lot of items that were not changed got reimported.

• Locks gets automatically extended after 12 hours

  When an import starts, a feed gets locked - to prevent running an import for the same feed simultaneously. When an import finishes, the feed gets unlocked.
  By default, a lock expires after 12 hours. This was problematic for imports that take longer to complete as it could cause concurrent imports to happen. To mitigate that issue, a lock can now get automatically extended after 12 hours. The new Feeds Lock service checks if an import is still active and if so, it extends the lock with another 12 hours.

  This is how the Feeds Lock service determines if an import is still active:

  • It checks if there any queue items for the feed in question. If so, the import is considered active.
  • It checks if there was import activity within the last hour by checking the last_activity variable in the key/value table. If so, the import is considered active.

  One caution: You may need to regular check your imports, because an import that got stuck on the queue may remain to be stuck.

• Feed objects are no longer serialized on the queue

  This saves memory and storage space in the database. Instead, a reference to the feed is saved on the queue.

  One caution for developers: if your module stores temporary data on a Feed object during an import, be sure to check if that still works as intended. That data could possibly get lost if an import takes multiple cron runs to complete.

• Revision support

  On the feed type you can configure to let Feeds create a new revision on each import.

Contributors (33)

MegaChriz, lind101, coaston, Nila Hyalij, Manoj Raj.R, carolpettirossi, unstatu, PunamShelke, Anybody, danharper, trickfun, webdrips, bharata, devad, irinaz, Project Update Bot, hershy.k, joco_sp, andreyjan, themodularlab, nikathone, mikran, bibo, darvanen, joshua.boltz, ramil g, andypost, joelpittet, andileco, ptmkenny, sker101, pixiekat, immaculatexavier

Changelog

Issues: 15 issues resolved.

Changes since 8.x-3.0-beta2:

Bug

• #3324981 by MegaChriz: BooleanTarget converts non-string values to FALSE
• #3132198 by MegaChriz: Clean queue tasks when unlocking a feed to prevent potential data loss.
• #3306211 by MegaChriz, lind101, coaston: Fixed PHP 8 deprecation: do not pass null to trim().
• #3193610 by MegaChriz: Fixed items getting unpublished/deleted unexpectedly: extend the feed lock after 12 hours in case an import appears to be still active.
• #3263626 by joco_sp, andreyjan, themodularlab: Fixed missing route error when installing feeds (occurred for some people).
• #3302254 by ptmkenny: Fixed namespacing class properties docblock in EntityProcessorBase.
• #3299584 by pixiekat, immaculatexavier: Use ConfigEntityStorageInterface in ExtraLinks class to fix an error when using webprofiler.

Feature

• #2907721 by MegaChriz, carolpettirossi, unstatu, PunamShelke, Anybody, danharper, trickfun, webdrips, bharata: Added Feeds Log module for tracking imports extensively (including items that failed to import).
• #2926744 by nikathone, mikran, MegaChriz, bibo, darvanen, danharper, joshua.boltz: Allow revisionable entities to be saved as new revisions.
• #3105322 by nikathone, sker101, MegaChriz, andileco: Added multivalue support for the Feeds item field so entities can be updated by multiple feeds without hashes getting overwritten.

Task

• #3322691 by MegaChriz: Refactor unit tests for targets
• #3319892 by Nila Hyalij, Manoj Raj.R, MegaChriz: Replaced README.txt with README.md
• #3313864 by devad, MegaChriz, irinaz: Added integration with the Ludwig module in order to be able to use Feeds on Drupal 10 without Composer.
• #3287478 by MegaChriz, Project Update Bot, hershy.k: Automated Drupal 10 compatibility fixes
• #3303166 by MegaChriz: Categorize source options on the mapping form for more clarity.
• #2978490 by ramil g, MegaChriz, andypost, joelpittet, andileco: Optimize Feeds queue: put reference to feed on queue task instead of the whole feed object.