If a Feeds Batch Import is interrupted for any reason (dropped network connection, timeout, server crash, etc), Imports can be left in a 'locked' state, so that the user can not try the import again.

Patch provides a one-button form for resetting the state of all feeds.

Comments

matt2000’s picture

StatusFileSize
new1.45 KB

Minor fix to previous patch. ($form shouldn't be passed by reference. Oops.)

johnv’s picture

This issue seems a nice work-around for the following problem: #1018520: Feeds Importer cannot be used anymore after first 'Import', then 'Delete items'. .
Haven't tested it yet.

axel.rutz’s picture

Status:Needs review» Reviewed & tested by the community

works like a charm:
* started import of huge file
* aborted it
* failed to start a new import => "import locked"
* "break lock"
* successfully started new import

so rtbc.

axel.rutz’s picture

could not apply last patch with ease so rerolled for convenience.

johnv’s picture

@axel.rutz, I deserve no credit - I merely crossposted the issues...

emackn’s picture

Status:Reviewed & tested by the community» Needs work

I don't think it's a good idea to update the status of every feed to unlock a single feed.

Jon Pugh’s picture

Assigned:matt2000» Jon Pugh
Status:Needs work» Needs review
StatusFileSize
new4.49 KB

Attached is a patch, totally reprogrammed to add this feature to all importers:

- Adds "Unlock" tab to each importer, right next to "delete items"
- Provides another feeds_acccess() type, "unlock". This gives each feed importer a new permission. To test you must check your user permissions, unless you are UID=1
- It uses db_update() to set state = FALSE. I do not know if this is the best way but there was no obvious way to save a feed source

alexmc’s picture

I've just tried the patch found in #7 because I have a feed stuck on 22% for the last two days.
Notes:
1) Why not set admin permissions to use this by default. (Ok - I added them manually)
2) I had to clear Drupal caches before the "unlock" tab appeared.

Unfortunately it tells me "This Importer is not locked, therefor it cannot be unlocked."

Well - it is definitely stuck and not doing anything, and I can't restart it.... So if that isn't "locked" what is?

So looking at the patch and the rest of the code for me

$progress = $source->progressImporting();
if ($progress !== FEEDS_BATCH_COMPLETE) {
$form['submit']['#disabled'] = TRUE;
$form['submit']['#value'] =
t('Importing (@progress %)', array('@progress' => number_format(100 * $progress, 0)));
}

is happening AND
$progress = $source->progressClearing();
if ($progress == FEEDS_BATCH_COMPLETE) {
$form['source_locked'] = array(
'#type' => 'markup',
'#title' => t('Not Locked'),
'#tree' => TRUE,
'#markup' => t('This Importer is not locked, therefor it cannot be unlocked.'),
);

for the same importer... what gives? Why does the first say "progressImporting" and the patch say "progressClearing"?

Jason Jaeger Geoprobe’s picture

I also get "This Importer is not locked, therefor it cannot be unlocked." when using the patch in #7 when it is most definitely stuck.

Ace Cooper’s picture

The patch #7 let's me unlock only those feeds, where I try to delete imported items after a failed session.
So I can only unlock importers when the status of the feed goes as follows:

Importing - 85 % complete.
Deleting items - 0 % complete.

When I try to unlock the failed session that says "Importing - 85 % complete." the Unlock option is Disabled ("This Importer is not locked, therefor it cannot be unlocked.").

ethnovode’s picture

Patch in #7 is working for me. Thanks.

Rob_Feature’s picture

Status:Needs review» Reviewed & tested by the community

Super necessary addition to this module. Tested and had success with #7. With a few of us reporting success, I'm marking this as RTBC

that0n3guy’s picture

Status:Reviewed & tested by the community» Needs work

I'm with #9 and #10 on this one. I'm switching back to needs work. The title off this issue is "Add tool the reset stuck feeds". Some feeds are stuck, but not locked. This patch doesn't help with that.

Rob_Feature’s picture

I guess I'm interested to hear the difference between 'locked' and 'stuck' (such technical terms!). It seems to me that when a feed gets 'stuck' the module 'locks' it. I know thats not correct, but can we get some clarity on the difference?

that0n3guy’s picture

Well, I can tell you that my feed is stuck at 2% b/c of this issue: http://drupal.org/node/1029102. If I could re-run it, I could debug it easier.

Rob_Feature’s picture

I think we're doing something similar here that0n3guy because I was over on that issue with the same problem. I found that as long as the feed wasn't 'locked' (using this patch) then I could continue the import using cron (drush cron). Have you tried running cron to see if things advance at all?

that0n3guy’s picture

Yup, thats what that other issue is all about.... errors when running cron. It doesn't really have anything to do with this issue I think.

If a feed is "locked" OR if its having issues and erroring out when cron is ran, either way both cases need to be able to be reset

Rob_Feature’s picture

Title:Add tool to reset stuck feeds» Add tool to reset locked feeds
Status:Needs work» Reviewed & tested by the community

I think I'd disagree because this issue has the patch for the locked problem (whcih was the original intent of this post)...I'd suggest that this title changes to reflect the 'locked' fix since that's what everyone here is testing with these patches. I might suggest creating a new issue if your problem is different than this 'locked' stuff that's addressed here.

See the initial post:

If a Feeds Batch Import is interrupted for any reason (dropped network connection, timeout, server crash, etc), Imports can be left in a 'locked' state, so that the user can not try the import again.

Patch provides a one-button form for resetting the state of all feeds.

Changing the issue title and marking as reviewed by community.

franz’s picture

This looks like a formidable feature, and I agree with #18. Let's get this in and one way or the other, this will be useful when dealing with similar "stuck" problems which can go on in other issues.

franz’s picture

Status:Reviewed & tested by the community» Fixed

Comitted this.

Other "stuck-like" issues can be opened as spin-offs from this.

milesw’s picture

Assigned:Jon Pugh» Unassigned
Status:Fixed» Needs review
StatusFileSize
new1.02 KB

Umm, the patch that went in does not reset stuck imports, it only resets stuck "clears". Comments #8, #9, #10, #13 indicated that it was not working for stuck imports.

With the minor change in this patch it works for imports as well. Patch is against HEAD, of course.

franz’s picture

Status:Needs review» Fixed

Thanks for the addition, milesw, from what I read in the latest comments, it was a misunderstanding that lead me to believe those cases were of a different kind. This time the issue for locked is address, I believe.

milesw’s picture

@franz: Thanks for taking care of this. :)

Status:Fixed» Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

Flimmer1970’s picture

Component:Feeds Import» Code
Issue summary:View changes
Issue tags:+same problem again after dev version from 14.Oct.

no Patch from the patches above is working!

only workaround is directly in SQL:
update feeds_source set state = NULL WHERE id = 'xxx';
update feeds_source set fetcher_result = NULL WHERE id = 'xxx';

GuyPaddock’s picture

@Flimmer1970: We use this feature every day without issue. How can you say it's not working?