Provide one queue for all import related tasks

Problem/Motivation

Right now there are three separate queues for tasks related to importing, namely:

• feeds_feed_import
• feeds_feed_parse
• feeds_feed_process

This is problematic because that results into cron processing less of the import task as could be. Let me explain that. When queue tasks are performed during cron runs, cron loops through all queues. The order of the queues is determined based on the alphabetical order that the class names appear in. So first 'feeds_feed_parse' (FeedParse), then 'feeds_feed_process' (FeedProcess) and finally 'feeds_feed_import' (FeedRefresh). Each queue is only run once during each cron run. So when an import is queued (put in feeds_feed_import), the following happens during cron:

1. Queue 'feeds_feed_parse' runs. No items found.
2. Queue 'feeds_feed_process' runs. No items found.
3. Queue 'feeds_feed_import' runs. One item found. Performs a fetch operation. Queues a task of type 'feeds_feed_parse'.

And then, on the next cron, finally some items start to get imported, but only a limited number of items if the parser supports importing items in chunks, like the XML parser from Feeds extensible parser does, for example. So if the parser limit is 50, only 50 items will get imported on the second cron run. After parsing and processing a new task of type 'feeds_feed_import' is queued, which on their turn will queue a new task of type 'feeds_feed_parse'.

This way a huge number of items will take many cron runs to complete.

Proposed resolution

Instead of having three queues, I think it is better to have just one queue. This way multiple cycles can be run in one cron run. With local testing I could import about 450 items with one cron run.

To determine what needs to happen when the queue task is run an operation must be specified when queuing the task. With the changes happened in #2803795: Cron import does not work with parser that uses progress we already have two: 'begin' and 'resume'. This are all the operations that I identified:

• begin
• resume
• parse
• process
• finish

The 'finish' operation just checks if the import is finished and if not, it queues a new task with operation 'resume'. This is what the 'final' process queue task does now. See the code in \Drupal\feeds\Plugin\QueueWorker\FeedsParse:

// Add a final process queue item that finalizes the import.
$queue->createItem([$feed, $fetcher_result]);

Patch will follow.