With the current API, once a worker has claimed an item, all it can do is:

  • either process and delete it when it's done
  • or die if it cannot process it

But in the latter case the job won't be available for another worker until the item timeout expires.

The attached patch includes a releaseItem() method for this.

CommentFileSizeAuthor
#3 queueRelease1.patch1.18 KBfgm
queueRelease.patch721 bytesfgm
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

chx’s picture

Status: Needs review » Needs work

Please roll a patch with the -up parameters. (There is a handbook page) For reviewers: we discussed this in IRC and fgm claimed SQS can do a ChangeMessageVisibility(0) for release and beanstalkd has a release op. I am not so sure, the SQS docs say You could call ChangeMessageVisiblity with a value of two hours and the effective timeout would be two hours and 30 minutes. -- so calling it with 0 is a no-op?

Damien Tournoud’s picture

For what I understand, ChangeMessageVisiblity actually works like set_time_limit(): it simply sets the running timeout counter to the new value. In that sense, if your item is about to timeout, calling ChangeMessageVisiblity(300) actually "extends" its visibility timeout.

The example paragraph in ChangeMessageVisiblity is actually misleading.

fgm’s picture

FileSize
1.18 KB

Patch rerolled. For reference for reviewers:

fgm’s picture

Status: Needs work » Needs review

Forgot to set status.

chx’s picture

Status: Needs review » Reviewed & tested by the community

I am fine with this after all. Given that there is no ordering guarantee if a queue implementation cant do this, there is no harm, after all and it can be useful.

Dries’s picture

What about DamZ's comment in #2? Is there a documentation improvement required?

fgm’s picture

I think that the documentation DamZ is referring to is the one provided by Amazon on their site.

In our case, releasing would be performed by a SQS implementation of DrupalQueue by setting visibility to 0, anyway.

Dries’s picture

Status: Reviewed & tested by the community » Fixed

Thanks for clarifying. Committed to CVS HEAD. Thanks.

yched’s picture

Just a note that this broke #629794: Scaling issues with batch API, which lies in the RTBC queue for a month.
No harsh feelings, but before updating the patch for the 6th time :-p, I'd like a word of confirmation from drieschick that it has a chance to get in.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.