Problem/Motivation

We have a Queue topic in Drupal 8:
https://api.drupal.org/api/drupal/core%21core.api.php/group/queue/8.4.x
(if this link is broken, go to Topics in the sidebar and find the Queue topic).

It currently says nothing about how to get a queue to be processed automatically in Cron.

Also, the plugin annotation class and hook related to this (QueueWorker annotation class and hook_queue_info_alter() ) are not listed as @ingroup for this topic, so they do not show up in the Functions and Classes lists on the topic page.

Proposed resolution

a) Add a section to the Queue topic about how to make a queue processed automatically in Cron. The main points:
- A queue can be set up to be processed in Cron by defining a QueueWorker plugin.
- This plugin needs to have \Drupal\Core\Annotation\QueueWorker annotation on it
- See the docs for that annotation class for details.

b) Make sure that the QueueWorker annotation class, Drupal\Core\Queue\QueueWorkerInterface, Drupal\Core\Queue\QueueWorkerBase and hook_queue_info_alter() both have lines in their doc blocks saying:
@ingroup queue

c) Make the Queue topic and hook_cron() have @see references to each other:
@see queue [in the hook]
@see hook_cron() [in the Queue topic]

Remaining tasks

Make a patch.

User interface changes

None.

API changes

None.

Beta eval

This is just API docs, so is unfrozen in Beta.

CommentFileSizeAuthor
#38 interdiff_2264943_37-38.txt2.15 KB_pratik_
#38 2264943-38.patch3.18 KB_pratik_
#37 2264943-37.patch2.94 KBNikhil_110
#36 2264943-nr-bot.txt153 bytesneeds-review-queue-bot
#25 interdiff-2264943-17-25.txt3.15 KBFeyP
#25 2264943-25-queue-api-topic-cron.patch3.79 KBFeyP
#17 2264943-17-queue-api-topic-link.patch907 bytesWeilinggu
#9 interdiff-2264943-6-9.txt550 byteser.pushpinderrana
#9 2264943-9-queue-api-topic-link.patch1.27 KBer.pushpinderrana
#6 2264943-queue-api-topic-link.patch1.2 KBgauravkhambhala
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

jhodgdon’s picture

jhodgdon
she/her
Primary language English
CreditAttribution: jhodgdon commented

It's not just linking... It seems like the Queue Operations topic needs to mention that before you can create a queue, you also need to declare it using hook_queue_info()? I guess, anyway?

Hm... hook_queue_info() says:

You can create queues, add items to them, claim them, etc without declaring the queue in this hook if you want, however, you need to take care of processing the items in the queue in that case.

And then later on, under Return value, it says:

If the cron key is not defined, the queue will not be processed by cron, and must be processed by other means.

So I think in the Queue Operations topic, we should say something about when you do and don't need to use hook_queue_info(), and what the consequences are. And definitely hook_queue_info(), not to mention the basic Queue classes and interfaces, should have @ingroup queue added to their doc blocks.

joachim’s picture

joachim CreditAttribution: joachim commented

> before you can create a queue, you also need to declare it using hook_queue_info

No, that's not the case. It's only if you want cron to run your queue for you.

gauravkhambhala’s picture

gauravkhambhala CreditAttribution: gauravkhambhala commented
Assigned: Unassigned » gauravkhambhala
gauravkhambhala’s picture

gauravkhambhala CreditAttribution: gauravkhambhala commented

The first link says "Sorry, api/drupal/core!includes!common.inc/group/queue/8 cannot be found". Could you pont out right link?

joachim’s picture

joachim CreditAttribution: joachim commented

It's been moved: https://api.drupal.org/api/drupal/core!modules!system!core.api.php/group...

gauravkhambhala’s picture

gauravkhambhala CreditAttribution: gauravkhambhala commented
Status: Active » Needs review
FileSize
2264943-queue-api-topic-link.patch1.2 KB

Added @ingroup queue to the functions. Please review the patch.

Also added @see links to hook_cron function. Let me know if that was required, otherwise I will remove that change and upload the patch again.

jhodgdon’s picture

jhodgdon
she/her
Primary language English
CreditAttribution: jhodgdon commented
Status: Needs review » Needs work

I do not think a @see in the hook_cron() documentation to callback_queue_worker() is really a good idea -- that seems like something only used by hook_queue_info() right?

By the way, you can also do
@see queue
and it will make a link to the Queue topic.

So... This patch is mostly good, but... I think we should do a bit more:

- Probably the Queue topic needs an @see to hook_cron()
- Probably hook_cron() needs an @see to "queue"... and probably also to hook_queue_info(), but not hook_queue_info_alter() or the callback function?
- Probably hook_queue_info() should have an @see to hook_cron()... oh, it already does. That's good. :)

joachim’s picture

joachim CreditAttribution: joachim commented

I don't think you need the @see to hook_queue_info() at the top of the patch, as it's in the doc text directly above it. As for the @see hook_queue_info_alter(), that's not needed either, as you can discover that exists when you go read about hook_queue_info().

er.pushpinderrana’s picture

er.pushpinderrana CreditAttribution: er.pushpinderrana commented
Status: Needs work » Needs review
FileSize
2264943-9-queue-api-topic-link.patch1.27 KB
interdiff-2264943-6-9.txt550 bytes

Moving forward to this, please review updated patch.

jhodgdon’s picture

jhodgdon
she/her
Primary language English
CreditAttribution: jhodgdon commented
Status: Needs review » Needs work

OK...

I still think we need this from #7:
- Probably the Queue topic needs an @see to hook_cron()

Actually, the Queue topic doesn't even talk about hook_queue_info in the text. Should it?

joachim’s picture

joachim CreditAttribution: joachim commented

I don't think the Queue topic should @see to hook_cron(). It should mention hook_queue_info() in the text though. At that point I suppose it could mention hook_cron(), but it's not really relevant, as it's not hook_cron() that works on queues declared by hook_queue_info() -- it's the base cron handling in system module.

jhodgdon’s picture

jhodgdon
she/her
Primary language English
CreditAttribution: jhodgdon commented

Well I don't think it says anything at all about cron now. Shouldn't it?

joachim’s picture

joachim CreditAttribution: joachim commented

The gist to convey is "if you want a queue to be worked on automatically by Drupal during cron runs, you need to declare it with hook_queue_info()". So yes, probably.

jmarkel’s picture

jmarkel CreditAttribution: jmarkel as a volunteer commented
Assigned: gauravkhambhala » Unassigned
Issue summary: View changes
Issue tags: -Novice +Needs issue summary update, +Needs reroll

Updated tags - needs re-roll and an updated issue summary.

I am removing the Novice tag from this issue because it appears to be a significant effort for anyone with no familiarity with the Queue system. It could be done by a novice but may take more time to read up on and digest the Drupal queueing system..

I’m using this documentation as a source: https://www.drupal.org/core-mentoring/novice-tasks#avoid

jmarkel’s picture

jmarkel CreditAttribution: jmarkel as a volunteer commented
Issue summary: View changes
Weilinggu’s picture

Weilinggu CreditAttribution: Weilinggu commented

I am at Drupalconla. I am working on this issue right now.

Weilinggu’s picture

Weilinggu CreditAttribution: Weilinggu commented
FileSize
2264943-17-queue-api-topic-link.patch907 bytes

hook_cron and hook_queue_info_alter are moved to core.api.php. Link has been added. Uploaded the patch here.

Can't find callback_queue_worker and hook_queue_info using grep in D8.

Weilinggu’s picture

Weilinggu CreditAttribution: Weilinggu commented
Status: Needs work » Needs review
jhodgdon’s picture

jhodgdon
she/her
Primary language English
CreditAttribution: jhodgdon commented
Issue summary: View changes
Status: Needs review » Needs work
Issue tags: -Needs issue summary update, -Needs reroll

Thanks for the new patch! You're right, things have changed in Drupal 8 since this issue was first filed.

So. It appears that in Drupal 8, instead of having hook_queue_info(), we now have a QueueWorker plugin annotation. So the way you say "This queue needs to run in Cron" is you define a class and give it QueueWorker annotation.
https://api.drupal.org/api/drupal/core!lib!Drupal!Core!Annotation!QueueW...

The Queue topic still doesn't say anything about this, and it should:
https://api.drupal.org/api/drupal/core!modules!system!core.api.php/group...

Updating the issue summary. We just need a bit more added to the patch, which is good as far as it goes.

joachim’s picture

joachim CreditAttribution: joachim at Torchbox commented

> So the way you say "This queue needs to run in Cron" is you define a class and give it QueueWorker annotation.

You probably also want to have it inherit from Drupal\Core\Queue\QueueWorkerBase, and if not, implement \Drupal\Core\Queue\QueueWorkerInterface.

jhodgdon’s picture

jhodgdon
she/her
Primary language English
CreditAttribution: jhodgdon commented

Definitely, and the base class and interface should also have an @ingroup so they appear in this topic.

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

FeyP’s picture

FeyP CreditAttribution: FeyP as a volunteer and at werk21 commented
Issue summary: View changes
Status: Needs work » Needs review
FileSize
2264943-25-queue-api-topic-cron.patch3.79 KB
interdiff-2264943-17-25.txt3.15 KB

I made some changes to the issue summary:

  • Updated the link to Queue API topic in the issue summary.
  • Updated proposed resolution section b) to include QueueWorkerInterface and QueueWorkerBase per comment 21.

The previous patch basically implemented some parts of the changes proposed in b) and c). Attached is a new patch against 8.4.x. Changes from previous patch:

  • core.api.php moved to /core/core.api.php
  • Removed an empty line inserted before opening php tag of core.api.php
  • Added @ingroup queue to Drupal\Core\QueueWorker\QueueWorkerInterface and Drupal\Component\Annotation\Plugin to complete b).
  • Split existing documentation in Queue API topic into different sections.
  • Added a section to Queue API topic as proposed in a).
  • Added @see hook_cron() to Queue API topic to complete c).

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.6 was released on August 2, 2017 and is the final full bugfix release for the Drupal 8.3.x series. Drupal 8.3.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.4.0 on October 4, 2017. (Drupal 8.4.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.4.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.4 was released on January 3, 2018 and is the final full bugfix release for the Drupal 8.4.x series. Drupal 8.4.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.5.0 on March 7, 2018. (Drupal 8.5.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.5.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.6 was released on August 1, 2018 and is the final bugfix release for the Drupal 8.5.x series. Drupal 8.5.x will not receive any further development aside from security fixes. Sites should prepare to update to 8.6.0 on September 5, 2018. (Drupal 8.6.0-rc1 is available for testing.)

Bug reports should be targeted against the 8.6.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

apaderno’s picture

apaderno
he/him
Primary language Italian
Location Brescia, 🇮🇹 🇪🇺
CreditAttribution: apaderno as a volunteer commented
Version: 8.6.x-dev » 8.9.x-dev

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

needs-review-queue-bot’s picture

needs-review-queue-bot CreditAttribution: needs-review-queue-bot as a volunteer commented
Status: Needs review » Needs work
FileSize
2264943-nr-bot.txt153 bytes

The Needs Review Queue Bot tested this issue. It either no longer applies to Drupal core, or fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

Apart from a re-roll or rebase, this issue may need more work to address feedback in the issue or MR comments. To progress an issue, incorporate this feedback as part of the process of updating the issue. This helps other contributors to know what is outstanding.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

Nikhil_110’s picture

Nikhil_110 CreditAttribution: Nikhil_110 at Srijan | A Material+ Company commented
FileSize
2264943-37.patch2.94 KB

Attached patch against drupal 10.1.x

_pratik_’s picture

_pratik_ CreditAttribution: _pratik_ as a volunteer and at Specbee for Drupal India Association commented
Status: Needs work » Needs review
FileSize
2264943-38.patch3.18 KB
interdiff_2264943_37-38.txt2.15 KB
joachim’s picture

joachim CreditAttribution: joachim as a volunteer commented
Status: Needs review » Reviewed & tested by the community

LGTM. Thanks!

larowlan’s picture

larowlan
Location 🇦🇺🏝.au GMT+10
CreditAttribution: larowlan at PreviousNext commented

Updating credits

longwave’s picture

longwave
he/him
Primary language English
Location UK
CreditAttribution: longwave at Full Fat Things commented
Status: Reviewed & tested by the community » Fixed

Committed and pushed 4b781a54fa to 10.1.x and 98c56f0a20 to 10.0.x and d92acc38f2 to 9.5.x. Thanks!

  • longwave committed 98c56f0a on 10.0.x 
    Issue #2264943 by FeyP, er.pushpinderrana, _pratik_, gauravkhambhala,...

  • longwave committed 4b781a54 on 10.1.x 
    Issue #2264943 by FeyP, er.pushpinderrana, _pratik_, gauravkhambhala,...

  • longwave committed d92acc38 on 9.5.x 
    Issue #2264943 by FeyP, er.pushpinderrana, _pratik_, gauravkhambhala,...
luenemann’s picture

luenemann
Primary language German
Location Südbaden, Germany
CreditAttribution: luenemann at Equinoxe GmbH commented
Version: 10.1.x-dev » 9.5.x-dev

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.