FeedsLockException when importing feeds

I am having the same issue when trying to import small number of feed items. But if i try to import it again it will import the second time. May be it is something to do with cache somewhere.

I have done some further research and found this error in my Apache error log -> Unsupported operand types in /home/public_html/sites/all/modules/feeds/includes/FeedsConfigurable.inc on line 149. Now, page 149
has this statement>

/**
* Implements getConfig().
*
* Return configuration array, ensure that all default values are present.
*/

147 public function getConfig() {
148 $defaults = $this->configDefaults();
149 return $this->config + $defaults;
}
Any idea?

Running cron every minute solved my problem. I don,t know if that is sustainable in the long run but it is working for now. Therefore, i think it is something to do with timing.

Also if you enabled the internal cron job disable it and use external cron.

This happened to me a few times and I fixed it by deleting the semaphore row associated with the importer.

Patch worked for me!

Number 5 seems to help, I can now at least start import jobs that would refuse to work because of the "cannot acquire lock" error.
However, when these jobs run they fail quickly and I have to manually unlock and restart them over and over. Again throwing the "cannot acquire lock" message.

Scratch my comment in #7, it turned out that was a server issue on my end.

Patch #5 worked like a charm.

This is not a bad idea, but we should just load the source and call FeedsSource::releaseLock().

AlfTheCat, what server issue did you have because i can't reproduce it on my local machine and it is happening on development server.

@michiellucas,

It could be a hardware issue. Is there a big difference in specs between your dev and production site? Or perhaps php.ini resource variables?

Hello,
I am still seeing this issue even after applying the patch. I am also noticing a pattern that it does 3 or 4 row imports before giving this error.

Attaching snapshot here
http://i.imgur.com/NAVnLkj.png

Any other clues or places I should look at? Also there is nothing added to the site log. I have disabled the site cron as well.
Anything we can do at the database level as well?

Thanks.

Hi,
Attaching the file here supporting the comment I made above in #12

Opps, ignore previous patch.

With #18 applied, I can still produce the error.

My situation might be unique. The error occurs sporadically when loading a page that calls this code:

  $importer_id = 'blocked_' . $feed_type . "s";
  $source = feeds_source($importer_id);

  // Override configurations...
  $config = $source->getConfig();

  $config['FeedsHTTPFetcher']['source'] = $url;
  // hide/show the batch api progress bar
  $config['process_in_background'] = FALSE;
  $source->setConfig($config);
  $source->save();
  //$source->startImport();
  while (FEEDS_BATCH_COMPLETE != $source->import()) {
    ;
  }

When the error occurs I immediately query semaphore and the table is empty.

I think I fixed my problem by using $source->startImport() and not bypassing the batch api with that while loop. import() invokes releaseLock().

Patch worked for me, though there are two places where "Ulock" needs to be "Unlock"

+    drupal_set_message(t('This feed is currently locked. Use the <a href="@url">Ulock</a> tab to unlock it.', array('@url' => url('import/' . $importer->id . '/unlock'))), 'warning', FALSE);

+    drupal_set_message(t('This feed is currently locked. Use the <a href="@url">Ulock</a> tab to unlock it.', array('@url' => url('node/' . $node->nid . '/unlock'))), 'warning', FALSE);

Closed #2417433: background process - import finishes and starts again - Cannot acquire lock error and #2659194: "Cannot acquire lock for source" when starting XML feed import as duplicates.

My small web team just lost it's Drupal developer, so I'm on my own trying to work with this issue. I have a lot of front-end Drupal work and have built a number of sites, but I'm not a coder. From reading this page, some form of this problem goes back four years. Is there an update planned for the Feeds module to correct this or is this something taking a back seat to Drupal 8 development? Hacking the code is not something I have the expertise to do.

Thanks.

@fujishooter
This issue is not on my list of fixing this for D7. You could try the patch from #18 and report back if that fixes the problem for you. According to #21 the patch would only need spelling corrections.

An automated test that would demonstrate the issue would help with getting this in.

Fixed the spelling from #18 and uploading corrected patch.

Trying the patch again after failing from #25.

Setting to Needs review

Re-created the patch with PHPStorm from #18 with the spelling corrected as indicated in #21.

Trying patch testing again.

Re-created the patch from #18 with the spelling corrected as indicated in #21.

Switching to needs review.

Patch #33 works for me.

Can we commit this?

@chegor
If you (or someone else) can provide the steps to reproduce this issue or - even better - write an automated test that demonstrates the issue, I will give this issue more priority.

Hi,

This patch is also working for us.
I've updated the patch to get it working with the latest dev version.

Thanks.

I ran into this problem recently after updating cURL and Feeds, and it seems like there's different issues here that people are running into. The patch fixes the problem with using Unlock in the Feeds UI, but it doesn't fix what's causing the lock in the first place.

I assume it's a variety of different issues causing locks, but for me the fix seems to be disabling cURL altogether in the feeds_http_request function within the includes/http_request.inc file.

I should also mention it only seem to happen for certain domains, most work fine.

Update: Seems to be an issue with Apache and cURL and not an issue with Drupal.

I am having the same issue (cannot acquire lock for source) but with a feed importer and SQL parser (using Feeds SQL). It seems like the code is looking for the table semaphore in my external database and not inside the drupal database. Now my automatic import is broken and even manually I can't get my data. Any idea here? Should I open a new issue? Thanks in advance.

Error Message (partially) after launching manually the import : An AJAX HTTP Error occurred. [...] Base table or view not found: 1146 Table 'my_external_database.semaphore' doesn't exist in ...

It was an error in a Rule fired during the Feeds import. The import stopped but the active database was always the external one. I guess this explains why the module tried to access a Drupal table in my external database. As soon as I modified the Rule, the import worked well until the end and I got no more semaphore error message.

rerolled patch

I'm still wondering what the patches in this issue are trying to solve. It's clear that it's about a locked feed, possibly caused by a crashed import. I don't understand however why the existing unlock functionality isn't working. Has it something to do with the semaphore table? Can someone show what's in their semaphore table when this issue occurs?

1. +++ b/feeds.pages.inc
   @@ -85,6 +85,13 @@ function feeds_import_form(array $form, array &$form_state, FeedsImporter $impor
   +  $progress_importing = $source->progressImporting();
   +  $progress_clearing = $source->progressClearing();
   ...
   +  $locked = $locked && $progress_importing === FEEDS_BATCH_COMPLETE;
   +  $locked = $locked && $progress_clearing === FEEDS_BATCH_COMPLETE;
   
   @@ -196,12 +211,23 @@ function feeds_import_tab_form($form, &$form_state, $node) {
   +  $progress_importing = $source->progressImporting();
   +  $progress_clearing = $source->progressClearing();
   ...
   +  $locked = $locked && $progress_importing === FEEDS_BATCH_COMPLETE;
   +  $locked = $locked && $progress_clearing === FEEDS_BATCH_COMPLETE;
   
   @@ -312,7 +344,12 @@ function feeds_unlock_tab_form($form, &$form_state, FeedsImporter $importer = NU
   +  $locked = $locked || $source->progressImporting() !== FEEDS_BATCH_COMPLETE;
   +  $locked = $locked || $source->progressClearing() !== FEEDS_BATCH_COMPLETE;
   

   I don't understand why the import/clear progress need to be checked for the complete status. Can somebody explain?

2. +++ b/feeds.pages.inc
   @@ -85,6 +85,13 @@ function feeds_import_form(array $form, array &$form_state, FeedsImporter $impor
   +  $locked = !lock_may_be_available("feeds_source_{$source->id}_{$source->feed_nid}");
   
   @@ -196,12 +211,23 @@ function feeds_import_tab_form($form, &$form_state, $node) {
   +  $locked = !lock_may_be_available("feeds_source_{$importer_id}_{$node->nid}");
   
   @@ -312,7 +344,12 @@ function feeds_unlock_tab_form($form, &$form_state, FeedsImporter $importer = NU
   +  $locked = !lock_may_be_available("feeds_source_{$source->id}_{$source->feed_nid}");
   

   This repeats a lot of complexer looking code. I think it's better to have an isLocked() method in the FeedsSource class with this code.

3. +++ b/feeds.pages.inc
   @@ -110,21 +117,29 @@ function feeds_import_form(array $form, array &$form_state, FeedsImporter $impor
   +    drupal_set_message(t('This feed is currently locked. Use the <a href="@url">Unlock</a> tab to unlock it.', array('@url' => url('import/' . $importer->id . '/unlock'))), 'warning', FALSE);
   
   @@ -196,12 +211,23 @@ function feeds_import_tab_form($form, &$form_state, $node) {
   +    drupal_set_message(t('This feed is currently locked. Use the <a href="@url">Unlock</a> tab to unlock it.', array('@url' => url('node/' . $node->nid . '/unlock'))), 'warning', FALSE);
   

   This line is too long. Insert a new line after "array(" and format like elsewhere in the code where a new line is followed after "array(".

4. +++ b/feeds.pages.inc
   @@ -110,21 +117,29 @@ function feeds_import_form(array $form, array &$form_state, FeedsImporter $impor
   +    $form['actions']['submit']['#value'] = t('Clearing (@progress %)',
   +      array('@progress' => number_format(100 * $progress_clearing, 0)));
   ...
   +    $form['submit']['#value'] =
   +      t('Importing (@progress %)', array('@progress' => number_format(100 * $progress_importing, 0)));
   
   @@ -196,12 +211,23 @@ function feeds_import_tab_form($form, &$form_state, $node) {
   +    $form['actions']['submit']['#value'] = t('Clearing (@progress %)', ¶
   +      array('@progress' => number_format(100 * $progress_clearing, 0)));
   
   @@ -209,6 +235,12 @@ function feeds_import_tab_form($form, &$form_state, $node) {
   +    $form['actions']['submit']['#value'] =
   +      t('Importing (@progress %)', array('@progress' => number_format(100 * $progress_importing, 0)));
   

   Please follow the same formatting as elsewhere in the code: only a new line after "array(" and the two closing brackets also on anew line.

5. The patch is failing tests. But before I can tell if the code is wrong or the test is wrong, I need to know what the problem is that is being solved here.

I dont know if this can help: This problem happen to me in a multisite installation, same codebase and different databases.
The import exists in each site, is called in the same way and possibly have the same ID. If I fire the import in the site A, and try to fire the same in the site B, the problem appears. I cant execute the same feed in the other sites mean while one of the sites is doing the process. After have been finished, the lock gets undone.

This is the validation:

  protected function acquireLock() {
    if (!lock_acquire("feeds_source_{$this->id}_{$this->feed_nid}", 60.0)) {
      throw new FeedsLockException(t('Cannot acquire lock for source @id / @feed_nid.', array('@id' => $this->id, '@feed_nid' => $this->feed_nid)));
    }
  }

In this case, there is a way of instanciate the feeds_source_@id dependant to site variables?

@Freddy Rodriguez
Do you have shared tables between these two sites? Because the information about whether or not a feed is locked is stored in the database table ‘semaphore’.
See lock_acquire().