Problem/Motivation

hook_queue_info provides a way to register a callback for doing work on a queue item.
The need for discovery and instantiation, in addition to swappability, makes it an ideal candidate for a plugin.

Proposed resolution

Replace hook_queue_info() with an annotated class to reside in \Drupal\*\Plugin\QueueWorker

See the \Drupal\Core\Annotation\QueueWorker class for the docs.

Remaining tasks

User interface changes

API changes

Replace hook_queue_info() with an annotated class
hook_queue_info_alter is unchanged.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

tim.plunkett’s picture

Status: Active » Needs review
FileSize
18.54 KB

Status: Needs review » Needs work
Issue tags: -Plugin system, -Plugin-conversion

The last submitted patch, queue-2038275-1.patch, failed testing.

tim.plunkett’s picture

Status: Needs work » Needs review
Issue tags: +Plugin system, +Plugin-conversion

#1: queue-2038275-1.patch queued for re-testing.

tim.plunkett’s picture

Priority: Normal » Major
Issue tags: +Approved API change
dawehner’s picture

Issue tags: -Approved API change
+++ b/core/lib/Drupal/Core/Annotation/QueueItem.phpundefined
@@ -0,0 +1,60 @@
+class QueueItem extends Plugin {

As written one day in IRC the name is quite odd. If nothing helps choose QueuePlugin

tim.plunkett’s picture

Status: Needs review » Needs work
Issue tags: +Approved API change

Yes, thanks for reminding me.

tim.plunkett’s picture

Status: Needs work » Needs review
FileSize
9.24 KB
18.72 KB

Let's just go with that.

dawehner’s picture

+++ b/core/core.services.ymlundefined
@@ -175,6 +175,9 @@ services:
+  plugin.manager.queue.item:

I guess we should also rename the plugin manager service then.

+++ b/core/lib/Drupal/Core/Queue/QueuePluginManager.phpundefined
@@ -0,0 +1,46 @@
+  public function __construct(\Traversable $namespaces, ModuleHandlerInterface $module_handler) {
+    parent::__construct('QueuePlugin', $namespaces, array(), 'Drupal\Core\Annotation\QueuePlugin');

No caching?

tim.plunkett’s picture

Assigned: tim.plunkett » Unassigned
Issue tags: -Approved API change
dawehner’s picture

Issue tags: +Approved API change

I guess removing the tag was not intended.

tim.plunkett’s picture

Issue tags: -Approved API change

No, it was incorrectly tagged. No one actually approved this, I was mistaken.

YesCT’s picture

Issue summary: View changes
Status: Needs review » Needs work
Issue tags: +Needs reroll
Sam Hermans’s picture

Assigned: Unassigned » Sam Hermans

I'll check this out

Sam Hermans’s picture

Assigned: Sam Hermans » Unassigned
Status: Needs work » Needs review
Issue tags: -Needs reroll
FileSize
20.14 KB

Re rolled against latest HEAD

dawehner’s picture

  1. +++ b/core/includes/common.inc
    @@ -3203,8 +3203,8 @@ function drupal_cron_run() {
    +  $manager = Drupal::service('plugin.manager.queue.item');
    

    This requires "\Drupal" now.

  2. +++ b/core/lib/Drupal/Core/Queue/QueuePluginInterface.php
    @@ -0,0 +1,22 @@
    + * @todo.
    
    +++ b/core/lib/Drupal/Core/Queue/QueuePluginBase.php
    @@ -0,0 +1,17 @@
    + * @todo.
    
    +++ b/core/lib/Drupal/Core/Queue/QueuePluginInterface.php
    @@ -0,0 +1,22 @@
    +  /**
    +   * @param mixed $data
    +   */
    

    Let's ensure to not forget this todo before commit

Status: Needs review » Needs work

The last submitted patch, 14: queue-2038275-14.patch, failed testing.

The last submitted patch, 14: queue-2038275-14.patch, failed testing.

mr.baileys’s picture

  • +++ b/core/lib/Drupal/Core/Queue/QueuePluginManager.php
    @@ -0,0 +1,46 @@
    +    parent::__construct('QueuePlugin', $namespaces, array(), 'Drupal\Core\Annotation\QueuePlugin');
    

    The parent constructor (DefaultPluginManager) only has three parameters, the array() here is wrong.

  • Re-rolled, since drupal_cron_run has been removed, and Cron is now a service (https://drupal.org/node/2181921).
  • Added basic documentation to replace the @todo's (mentioned in #15).
  • Changed Drupal::service('plugin.manager.queue.item'); to use \Drupal (reported in #15).
mr.baileys’s picture

Status: Needs work » Needs review

To the testbot!

Status: Needs review » Needs work

The last submitted patch, 18: 2038275-18-queue-plugin.patch, failed testing.

mr.baileys’s picture

Status: Needs work » Needs review
FileSize
20.42 KB
+++ b/core/lib/Drupal/Core/Queue/QueuePluginManager.php
@@ -0,0 +1,46 @@
+    parent::__construct('QueuePlugin', $namespaces, 'Drupal\Core\Annotation\QueuePlugin');

Should be 'Plugin/QueuePlugin' as $subdir.

Status: Needs review » Needs work

The last submitted patch, 21: 2038275-21-queue-plugin.patch, failed testing.

tim.plunkett’s picture

Assigned: Unassigned » tim.plunkett

Rerolling.

tim.plunkett’s picture

Status: Needs work » Needs review
FileSize
29.62 KB
19.45 KB

Updated for core changes, added an interface for the plugin manager, other fixes. Big interdiff.

neclimdul’s picture

After talking out loud in IRC I think my concern with this is currently the terminology. The plugin stuff we're doing isn't really connected to the QueueAPI, we're replacing the hook callback runner weirdness(which was sort of its own thing) with a runners based on plugins.

I think I would prefer this if the QueueManager was maybe QueueRunnerManager or QueueProcessorManager and if the surrounding implementation followed that terminology. That'll seperate it from the Queue and QueueItem concepts that already exist. I don't know what is the right term but I think either of those would be acceptable until I hear something better.

tim.plunkett’s picture

FileSize
17.12 KB
29.62 KB

Agreed, I really disliked the previous naming (QueuePlugin was such a cop-out)

EDIT: We discussed in IRC and landed on QueueWorker.

jibran’s picture

Status: Needs review » Reviewed & tested by the community

I think @neclimdul is happy know. I have seen him ++ @tim.plunkett in IRC. I reviewed the patch and didn't find anything objectionable so this is good to go.

  1. +++ b/core/lib/Drupal/Core/Queue/QueueWorkerBase.php
    @@ -0,0 +1,17 @@
    +abstract class QueueWorkerBase extends PluginBase implements QueueWorkerInterface {
    

    Why we need this class?

  2. +++ b/core/modules/locale/src/Plugin/QueueWorker/LocaleTranslation.php
    @@ -0,0 +1,120 @@
    +class LocaleTranslation extends QueueWorkerBase implements ContainerFactoryPluginInterface {
    

    Oh, I got it. This is the reason. :)

damiankloip’s picture

  1. +++ b/core/modules/aggregator/src/Plugin/QueueWorker/AggregatorRefresh.php
    @@ -0,0 +1,31 @@
    +  public function processItem($data) {
    +    if ($data instanceof FeedInterface) {
    

    We should really not do this, but this is not the fault of this issue.

  2. +++ b/core/modules/aggregator/src/Tests/AggregatorCronTest.php
    @@ -21,11 +21,11 @@ public function testCron() {
    -    $this->assertEqual(5, db_query('SELECT COUNT(*) FROM {aggregator_item} WHERE fid = :fid', array(':fid' => $feed->id()))->fetchField(), 'Expected number of items in database.');
    +    $this->assertEqual(5, db_query('SELECT COUNT(*) FROM {aggregator_item} WHERE fid = :fid', array(':fid' => $feed->id()))->fetchField());
    ...
    -    $this->assertEqual(0, db_query('SELECT COUNT(*) FROM {aggregator_item} WHERE fid = :fid', array(':fid' => $feed->id()))->fetchField(), 'Expected number of items in database.');
    +    $this->assertEqual(0, db_query('SELECT COUNT(*) FROM {aggregator_item} WHERE fid = :fid', array(':fid' => $feed->id()))->fetchField());
    ...
    -    $this->assertEqual(5, db_query('SELECT COUNT(*) FROM {aggregator_item} WHERE fid = :fid', array(':fid' => $feed->id()))->fetchField(), 'Expected number of items in database.');
    +    $this->assertEqual(5, db_query('SELECT COUNT(*) FROM {aggregator_item} WHERE fid = :fid', array(':fid' => $feed->id()))->fetchField());
    

    Any reason we are removing those messages? Just wondering.

alexpott’s picture

Status: Reviewed & tested by the community » Needs work

We need to bring this in line with the plugin api documentation efforts see \Drupal\Core\Annotation\Action for an example.

alexpott’s picture

  1. +++ b/core/lib/Drupal/Core/Queue/QueueWorkerBase.php
    @@ -0,0 +1,17 @@
    +/**
    + * Provides a basis for queue worker plugins.
    + */
    +abstract class QueueWorkerBase extends PluginBase implements QueueWorkerInterface {
    
    +++ b/core/lib/Drupal/Core/Queue/QueueWorkerInterface.php
    @@ -0,0 +1,23 @@
    +/**
    + * Defines an interface for queue workers.
    + */
    +interface QueueWorkerInterface extends PluginInspectionInterface {
    
    +++ b/core/lib/Drupal/Core/Queue/QueueWorkerManager.php
    @@ -0,0 +1,60 @@
    +/**
    + * Defines the queue worker manager.
    + */
    +class QueueWorkerManager extends DefaultPluginManager implements QueueWorkerManagerInterface {
    

    Needs docs update for the aforementioned plugin api docs standards.

  2. +++ b/core/modules/aggregator/aggregator.module
    @@ -138,22 +138,6 @@ function aggregator_cron() {
    -    'worker callback' => function (FeedInterface $feed) {
    

    Can remove use Drupal\aggregator\FeedInterface; from aggregator.module now

  3. +++ b/core/modules/locale/src/Plugin/QueueWorker/LocaleTranslation.php
    @@ -0,0 +1,120 @@
    +/**
    + * @QueueWorker(
    + *   id = "locale_translation",
    + *   title = @Translation("Update translations"),
    + *   cron = {"time" = 30}
    + * )
    + */
    +class LocaleTranslation extends QueueWorkerBase implements ContainerFactoryPluginInterface {
    

    Missing docs

  4. +++ b/core/modules/system/system.api.php
    @@ -157,35 +118,6 @@ function hook_queue_info_alter(&$queues) {
    - * @throws \Exception
    - *   The worker callback may throw an exception to indicate there was a problem.
    - *   The cron process will log the exception, and leave the item in the queue to
    - *   be processed again later.
    - * @throws \Drupal\Core\Queue\SuspendQueueException
    - *   More specifically, a SuspendQueueException should be thrown when the
    - *   callback is aware that the problem will affect all subsequent workers of
    - *   its queue. For example, a callback that makes HTTP requests may find that
    - *   the remote server is not responding. The cron process will behave as with a
    - *   normal Exception, and in addition will not attempt to process further items
    - *   from the current item's queue during the current cron run.
    

    Where has this documentation gone? I would expect it to be on the QueueWorkerInterface.

Also the issue summary could be better and maybe we need a change record (8.x to 8.x I think).

jhodgdon’s picture

alexpott asked me to help with the docs here.

What we need is docs like what was added for each child issue of #2269389: [meta] Make sure plugin developer info is discoverable. The child issues all have detailed instructions; one example: #2290281: Tip plugin classes need more docs links. It should take about 5 minutes to do...

tim.plunkett’s picture

Issue summary: View changes
Status: Needs work » Needs review
Issue tags: -Needs issue summary update
FileSize
31.35 KB
4.74 KB

Will work on a change notice now.

jhodgdon’s picture

The docs additions look good, at least as far as formatting and completeness. Not being involved in this issue, I haven't reviewed them for accuracy. Thanks!

tim.plunkett’s picture

Status: Needs review » Reviewed & tested by the community
catch’s picture

Status: Reviewed & tested by the community » Needs work

Couple of docs fixes. Looks good otherwise.

  1. +++ b/core/lib/Drupal/Core/Annotation/QueueWorker.php
    @@ -0,0 +1,65 @@
    + * the queue in this hook if you want, however, you need to take care of
    

    Not a hook any more.

  2. +++ b/core/lib/Drupal/Core/Queue/QueueWorkerInterface.php
    @@ -0,0 +1,45 @@
    +   *   callback is aware that the problem will affect all subsequent workers of
    

    method? worker?

tim.plunkett’s picture

Status: Needs work » Reviewed & tested by the community
FileSize
31.41 KB
2.63 KB

Thanks @catch, I fixed up the docs on both of those to be consistent with the changes in #33.

alexpott’s picture

Status: Reviewed & tested by the community » Fixed

Committed ac0cee8 and pushed to 8.0.x. Thanks!

  • alexpott committed ac0cee8 on 8.0.x
    Issue #2038275 by tim.plunkett, mr.baileys, Sam Hermans: Convert...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.