Here is mention of the lock renewal functionality that is jamming every now and then our feeds https://www.drupal.org/node/3363755

Problem/Motivation

When a queue worker dies, is killed, or times out mid-import, its queue item and the feed's semaphore lock are left behind. On subsequent cron runs, FeedsLockBackend::lockMayBeAvailable() sees that queue tasks still exist for the feed and extends the lock indefinitely (logging only "Lock feeds_feed:N got extended"). The feed stays locked forever, no further imports run, and there is no clear error to alert on.

The issue can be solved by hand with ´drush feeds:unlock [fid]´

Steps to reproduce

  1. Configure a feed with a queued (cron) import.
  2. Start an import so a feeds_feed_refresh: queue item is created.
  3. Kill the PHP worker mid-run (SIGKILL, pod restart, OOM, deploy).
  4. Wait past feeds.settings:lock_timeout (default 12 h).
  5. Observe on every subsequent cron: Lock feeds_feed:N got extended, feed remains locked, imported never updates.

Proposed resolution

Add a small StaleImportRecovery service that decides when a queued import has genuinely gone stale and resets it:

  • Record started (worker actually picked the item up) in a dedicated feeds_stale_recovery. keyvalue collection when FeedsExecutable::import() begins. Queued-but-not-yet-started items in a cron backlog are ignored.
  • isStale() = marker set AND queue tasks exist AND started + stale_import_timeout <= now AND no hasRecentProgress(stale_import_grace_period).
  • When FeedsLockBackend::lockMayBeAvailable() sees an expired lock, it calls StaleImportRecovery::recover() before falling back to the old "extend if queue tasks or recent progress" branch.
  • Recovery serialises via a short-lived feeds_stale_recovery: lock, has a reentrancy guard, then calls Feed::unlock() (which already handles queue/state/temp-file cleanup) and invokes a new hook_feeds_stale_import_recover($feed, $context) for downstream logging/alerting.
  • Two new feeds.settings keys with sane defaults: stale_import_timeout (7200 s) and stale_import_grace_period (3600 s).

Remaining tasks

  • Review, patch tidy-up.
  • Consider a settings-form UI for the two new values (currently config-only).
  • Consider adding a Drush command for on-demand recovery.
  • Document hook_feeds_stale_import_recover() in feeds.api.php.

User interface changes

None. Settings are managed via config only for now.

API changes

  • New service: feeds.stale_import_recovery (Drupal\feeds\StaleImportRecovery).
  • New hook: hook_feeds_stale_import_recover(FeedInterface $feed, array $context).
  • FeedsLockBackend::__construct() gains a StaleImportRecovery parameter (BC break for anyone directly instantiating it).

Data model changes

  • New config: feeds.settings:stale_import_timeout (int, default 7200), feeds.settings:stale_import_grace_period (int, default 3600).
  • New keyvalue collection: feeds_stale_recovery. with key started (int timestamp). Cleared on Feed::unlock() and after successful recovery.

Issue fork feeds-3625598

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

sceefo created an issue. See original summary.

sceefo’s picture

Status: Needs review » Needs work
sceefo’s picture

StatusFileSize
new24.38 KB

benstallings made their first commit to this issue’s fork.

benstallings’s picture

Version: 8.x-3.2 » 8.x-3.x-dev
Status: Needs work » Needs review