Here is mention of the lock renewal functionality that is jamming every now and then our feeds https://www.drupal.org/node/3363755
Problem/Motivation
When a queue worker dies, is killed, or times out mid-import, its queue item and the feed's semaphore lock are left behind. On subsequent cron runs, FeedsLockBackend::lockMayBeAvailable() sees that queue tasks still exist for the feed and extends the lock indefinitely (logging only "Lock feeds_feed:N got extended"). The feed stays locked forever, no further imports run, and there is no clear error to alert on.
The issue can be solved by hand with ´drush feeds:unlock [fid]´
Steps to reproduce
- Configure a feed with a queued (cron) import.
- Start an import so a feeds_feed_refresh: queue item is created.
- Kill the PHP worker mid-run (SIGKILL, pod restart, OOM, deploy).
- Wait past feeds.settings:lock_timeout (default 12 h).
- Observe on every subsequent cron: Lock feeds_feed:N got extended, feed remains locked, imported never updates.
Proposed resolution
Add a small StaleImportRecovery service that decides when a queued import has genuinely gone stale and resets it:
- Record started (worker actually picked the item up) in a dedicated feeds_stale_recovery. keyvalue collection when FeedsExecutable::import() begins. Queued-but-not-yet-started items in a cron backlog are ignored.
- isStale() = marker set AND queue tasks exist AND started + stale_import_timeout <= now AND no hasRecentProgress(stale_import_grace_period).
- When FeedsLockBackend::lockMayBeAvailable() sees an expired lock, it calls StaleImportRecovery::recover() before falling back to the old "extend if queue tasks or recent progress" branch.
- Recovery serialises via a short-lived feeds_stale_recovery: lock, has a reentrancy guard, then calls Feed::unlock() (which already handles queue/state/temp-file cleanup) and invokes a new hook_feeds_stale_import_recover($feed, $context) for downstream logging/alerting.
- Two new feeds.settings keys with sane defaults: stale_import_timeout (7200 s) and stale_import_grace_period (3600 s).
Remaining tasks
- Review, patch tidy-up.
- Consider a settings-form UI for the two new values (currently config-only).
- Consider adding a Drush command for on-demand recovery.
- Document hook_feeds_stale_import_recover() in feeds.api.php.
User interface changes
None. Settings are managed via config only for now.
API changes
- New service: feeds.stale_import_recovery (Drupal\feeds\StaleImportRecovery).
- New hook: hook_feeds_stale_import_recover(FeedInterface $feed, array $context).
- FeedsLockBackend::__construct() gains a StaleImportRecovery parameter (BC break for anyone directly instantiating it).
Data model changes
- New config: feeds.settings:stale_import_timeout (int, default 7200), feeds.settings:stale_import_grace_period (int, default 3600).
- New keyvalue collection: feeds_stale_recovery. with key started (int timestamp). Cleared on Feed::unlock() and after successful recovery.
| Comment | File | Size | Author |
|---|---|---|---|
| #3 | feeds-stale-import-recovery-3625598-1.patch | 24.38 KB | sceefo |
Issue fork feeds-3625598
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #2
sceefo commentedComment #3
sceefo commentedComment #6
benstallings commented