Use case:

  • The same node is updated numerous times per day (either manually or by Feeds import)
  • xmlsitemap_node_node_update() creates a new queue item each time the node is updated
  • xmlsitemap_node_cron doesn't have a chance to process and delete the queue items for whatever reason (error, cron timeout, etc)
  • Numerous queue items persist in the queue table for the same nodes, which creates a queue backlog which thwarts sitemap.xml from updating properly
  • After installing the very helpful queue_ui module, we discovered over 300,000 queue items for only a few hundred nodes of a particular content type. Yes, we need to figure out the underlying issue, but xmlsitemap doesn't need to create a new queue item for existing, unprocessed items.

    I propose checking for a queue item before creating it.

    Patch is on its way . . .

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

darkodev created an issue. See original summary.

darkodev’s picture

jasonmce’s picture

It is a good start, but it appears that an update to node 1 would match queued rows for any nid with a 1 in it (ex: 1234) and be excluded?
You probably want your condition to be the full data = 's:DIGITS_IN_NID:"NID";', but quote wrapping ":data" in your like condition should be just as safe.

darkodev’s picture

Good point, thanks.

Will refactor, test and upload a new patch.

I checked and the value for the like condition in my patch already contains the quotes.

':data' => '%"' . db_like($node->nid) . '"%'

The resultant query doesn't select anything except the nid in question.

eg.

select item_id from `queue` where name = 'xmlsitemap_node' and data like '%"1"%'
only selects nid 1

select item_id from `queue` where name = 'xmlsitemap_node' and data like '%"1234"%'
only selects nid 1234

Unless I'm missing something . . .

jasonmce’s picture

There it is! Apologies for the noise.

darkodev’s picture

@jasonmce thanks for second set of eyes

bricel’s picture

Adding support for the advanced queue module.

bricel’s picture

Making query condition more efficient. The nid is save as is, so need for %like%

bricel’s picture

bricel’s picture

Cleanup and fixes for patch #2 and #3

bricel’s picture

FileSize
1.42 KB

Fix patch.

bricel’s picture

FileSize
1.42 KB

Fix query from #5

bricel’s picture

FileSize
1.6 KB