Problem/Motivation
We are using 8.x-3.0 on a site that has in excess of a million entities, what this means is that sometimes we can cause our purge queue to balloon with a lot of data, it has on occasion shut itself down when it breaches the 100,000 item limit. Under these circumstances the purge queue then quickly grows and we have to empty the queue.
We noticed that when this happens if we attempt to empty the queue via the UI we quite often end up with our Database struggling with the mammoth task of performing a delete across hundreds of thousands of rows. We have instead resorted to running MySQL commands directly on the database to truncate the table.
Steps to reproduce
Configure a database queue and fill it with more than 100,000 records, preferably in excess of 500,000 to see the real performance hit of a delete statement. The AJAX will often time out and the database can struggle under severe load.
Proposed resolution
It seems like this could be done with a one-line change to the DatabaseQueue::deleteQueue() function and switch to use Connection::truncate() rather than Connection::delete(). A truncate is far more performant on large datasets as there are no row-level locks required to do so. Given that there are no arguments or conditions used when deleting the items in the queue, it seems safe to switch to a truncate.
Remaining tasks
User interface changes
None
API changes
None
Data model changes
None
| Comment | File | Size | Author |
|---|---|---|---|
| #2 | purge-truncate_not_delete-3178028-2.patch | 508 bytes | owenbush |
Issue fork purge-3178028
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #2
owenbush commentedAttached is a patch to use truncate rather than delete.
Comment #5
japerry