Problem/Motivation

I have an application with more than 300 (and counting) derivative queues, making Queue UI difficult to use.

Steps to reproduce

Create a derivative queue.

Proposed resolution

Group derivative queues in the UI under a base queue worker umbrella, and allow to be inspected and processed as a single queue.

Remaining tasks

User interface changes

API changes

Data model changes

Issue fork queue_ui-3446581

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

AaronBauman created an issue. See original summary.

aaronbauman’s picture

Issue summary: View changes

Not sure what I was looking at, but the item count for subqueues looks fine in fact.

voleger’s picture

Version: 3.1.x-dev » 3.2.x-dev
Status: Active » Postponed (maintainer needs more info)

Can you provide an example of a derivative queue definition for a test case?

aaronbauman’s picture

Here's the deriver i'm using for the curious:

/**
 * @file
 * Contains \Drupal\media_analysis\Plugin\Derivative.
 */

namespace Drupal\media_analysis\Plugin\Derivative;

use Drupal\Component\Plugin\Derivative\DeriverBase;
use Drupal\Core\Entity\EntityStorageInterface;
use Drupal\Core\Plugin\Discovery\ContainerDeriverInterface;
use Drupal\media_analysis\HookImplementations;
use Symfony\Component\DependencyInjection\ContainerInterface;

/**
 * Provides block plugin definitions for nodes.
 *
 * @see \Drupal\media_analysis\Plugin\QueueWorker\MediaAnalysisWorker
 */
class MediaAnalysisWorker extends DeriverBase implements ContainerDeriverInterface {

  /**
   * Constructor.
   *
   * @param \Drupal\Core\Entity\EntityStorageInterface $node_storage
   *   The node storage.
   */
  public function __construct(private readonly EntityStorageInterface $nodeStorage) {
  }

  /**
   * {@inheritdoc}
   */
  public static function create(ContainerInterface $container, $base_plugin_id) {
    return new static(
      $container->get('entity_type.manager')->getStorage('node')
    );
  }

  /**
   * {@inheritdoc}
   */
  public function getDerivativeDefinitions($base_plugin_definition) {
    $nodes = $this->nodeStorage->loadByProperties(['type' => 'media_channel']);
    foreach ($nodes as $node) {
      $this->derivatives[$node->id()] = $base_plugin_definition;
      $this->derivatives[$node->id()]['admin_label'] = t('Channel: ') . $node->label();
    }
    return $this->derivatives;
  }

}

This deriver then gets referenced in the @Cron annotation field of my existing cron queue worker:

/**
 * Defines 'media_analysis' queue worker.
 *
 * @QueueWorker(
 *   id = "media_analysis",
 *   title = @Translation("Media Analysis: Channels"),
 *   cron = {"time" = 60},
 *   deriver = "\Drupal\media_analysis\Plugin\Derivative\MediaAnalysisWorker"
 * )
 */

And I have another procedure to insert queue items with a queue name like media_analysis:$nid which then get processed by the queue worker, thanks to Derivative API.

The derivative is mostly useful so that I can more easily prevent duplicate queue items, but it breaks the Queue UI pretty badly.

I started looking at adapting Queue UI, but it's a bit of a can of worms - will require special handling for every route and callback that uses the {queueName} placeholder, for starters.
So IMO it's fine to leave this issue on Postponed, and I'll implement a custom UI for my special case.

voleger’s picture

Assigned: Unassigned » voleger
Status: Postponed (maintainer needs more info) » Needs work

So, you expect to see a separate view based on the selected base derivative worker definition, right? I see that as a separate route with the same table rener approach for the overview page, but queues are created based on the selected derivative.

I'll prepare the MR with the approach I see so you can try it out.

voleger’s picture

Assigned: voleger » Unassigned

@aaronbauman, please try patching the 3.2.x version of the queue_ui module with the diff of MR. Let me know whether it is a usable iteration for now.

I'm okay with the current implementation and will add the functional test coverage. However, I need feedback on the current version of the feature.

  • voleger committed 74971479 on 3.2.x
    Issue #3446581 by voleger, aaronbauman: Better support for Derivative...
voleger’s picture

Status: Needs work » Fixed

I added test coverage for grouping. It has to be stable. In case of any issues, create a follow-up.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.