Use support for running rebuild via queues

Patch for Drupal 8 but without #10 implementations.

+++ b/xmlsitemap.drush.inc
@@ -30,9 +30,94 @@ function xmlsitemap_drush_command() {
+    $query = \Drupal::entityQuery($entity_type_id);
+    $query->addTag('xmlsitemap_link_bundle_access');
+    $query->addTag('xmlsitemap_rebuild');
+    $query->addMetaData('entity_type_id', $entity_type_id);
+    $query->addMetaData('entity_info', $info);
+
+    if ($types = xmlsitemap_get_link_type_enabled_bundles($entity_type->id())) {
+      $bundle_name = $entity_type->getKey('bundle');
+      $query->condition($bundle_name, $types, 'IN');
+    }
+    else {
+      // If no enabled bundle types, skip everything else
+     continue;
+    }
+
+    $results = $query->execute();

This entityQuery might already run out of memory before even having the chance to process the chunks in batches.

Queuing got me as far as indexing 120k links now for that sitemap. Next: drush xmlsitemap-regenerate dies at following memory_limits: 512M, 1024M, 2048M, 4096M

(some) error messages:

Fatal error: Allowed memory size of 1073741824 bytes exhausted (tried to allocate 9201 bytes) in /docroot/core/lib/Drupal/Core/Database/Statement.php on line 59

Warning: Unknown: Drupal\Core\StreamWrapper\PublicStream::stream_write is not implemented! in Unknown on line 0

Fatal error: Allowed memory size of 2147483648 bytes exhausted (tried to allocate 8208 bytes) in /docroot/core/lib/Drupal/Core/Routing/CompiledRoute.php on line 146
Segmentation fault

Apart from taking more than half an hour it still doesn't finish. Anyways, at 4GB I gave up.

Re-roll against latest development version.

I'm a little surprised to see Drush command in this patch. I would think that the Drush core queue-run command would process the queue just fine.

I'm having similar results to Andre-B in #14. Trying to create a sitemap of ~120k nodes. I've increased the RAM of my VM to 4GB and set the PHP max_execution_time to unlimited. Both the UI and Drush sitemap builds give either 'memory exhausted' or 'timeout' errors. This is both with and without the patch in #15.

I'm seeing xmlsitemap:regenerate finish fine at the 8.x-1.2-alpha2 tag, but fail with memory errors at the latest 8.x-1.x. We have ~450,000 rows in xmlsitemap table.

I realize that this issue is wrt to rebuild, not regenerate, so that should probably be a separate issue, although perhaps related.

Reroll against current dev

Still having this issue on Drupal 9.3

Patch #21 is no longer applying.

I did an attempt to refactor the patch.

Refactoring of the patch mostly is about moving the additions from xmlsitemap.drush.inc to src/Commands/XmlSitemapCommands.php
But I didn't get it to work.

After that I tested the normal `drush xmlsitemap:rebuild` command which worked well for our scenario.
Number of records in xmlsitemap: 3400
Memory Limit setting: 256M

So the patch seems obsolete for our case.

I also did a test with the #21 patch on the 1.3.0 version of xmlsitemap in combination with drupal core 9.4.8. Expecting that it worked but in my case it didn't. The drush command `drush xmlsitemap-queue-rebuild` wasn't recognized.

Additionally the patch seems not to take entity types without bundles into account. Therefore the patch fails in that case on: $entity_type->getKey('bundle'). I partly fixed that with an $entity_type->hasKey('bundle') check.

Since the patch seemed already broken or at least not working for our scenario with the 1.3.0. version and a recent drupal core version. And since the already available `xmlsitemap-rebuild` command works fine in our scenario. I decided to not further investigate and/or refactor the patch for 1.4.0. With the WIP for 1.4.0 the drush command works, but in my case it didn't fill the queue table.

But maybe for future use or for someone else the WIP is usefull, so therefore it's added with this comment.
As said it isn't ready for (production) usage yet. To apply it as a patch you'd probably have to change the extension from .txt to .patch.

I still have an interest in this feature/patch. Hoping someone can pick this back up.