Add a non blocking batch runner

IMHO this sounds fantastic. For its use in the installer, but also in other cases, I think its important that it actually runs through, which is what may be hard to ensure. I guess you can already leave the page when a batch is running but now that it would be less evidently running, people may leave the page even more. That said, it would be fantastic if we would not block the user doing something just because we also need Drupal to do a lot of things in the background.

Also the tricky part about this may be that the installer language import updates default config en masse so we nee to be careful what other config saves we allow at the same time as they may go over each other :)

Yeah downloading the translation files could be very well parallelized, if d.o file server can handle the load :) Importing them I'm not sure can be parallelized since importing a .po file can have far reaching effects on dozens of config files at once. Then the importing next .po file is has effects on another dozen files, some overlap with the previous, etc.

I did some experiments with running queues through JS polling.
https://www.drupal.org/project/queue_manager

The module handles pure queues not batch operations but the approach is similar to what has been discussed above.

The queue status is just a render element that constantly gets queue size through ajax requests.

/**
 * Provides a render element to get queue size.
 *
 * Properties:
 * - #title: Element title.
 * - #queue_name: Queue name.
 * - #refresh: Refresh timeout in seconds.
 *
 * Usage Example:
 * @code
 * $build['example_status'] = [
 *   '#type' => 'queue_size',
 *   '#title' => 'Number of items',
 *   '#queue_name' => 'example',
 *   'refresh' => 1,
 * ];
 * @endcode
 */

And for running the queue there is another render element.

/**
 * Provides a render element to run a queue.
 *
 * Properties:
 * - #queue_name: Queue name to run.
 *
 * Usage Example:
 *
 * @code
 * $build['example_runner'] = [
 *   '#type' => 'queue_runner,
 *   '#queue_name' => 'example',
 * ];
 * @endcode
 */

For the Batch API we can create a similar couple of elements. BatchStatus (progress par) and BatchRunner. It would give us the following features that are not available in the current Batch API implementation.

1. Not blocking progress bar. It would be just an ordinary render element. It can emit some JS events when the batch is finished or failed allowing to block/unblock form submission.
2. Multiple progress bars for different batches on the same form.
3. Concurrency. In the above example if you add another queue_runner element the queue will be processed in two threads.

Generally speaking not every queue system has ability to get know a length of the queue (MQTT), moreover some of them give very approximate numbers (SQS/KAfka) so sometimes you need observability system to get this data

I created a module that allows to run tasks and display the operation status in non blocking mode. The approach is similar to what was described in #9. The progress bar is just a read-only indicator of the operation status. The queue runner is fully decoupled from page state.
The modules is not battle tested yet. The current development status is "working PoC".