Problem/Motivation

We are using 8.x-3.0 on a site that has in excess of a million entities, what this means is that sometimes we can cause our purge queue to balloon with a lot of data, it has on occasion shut itself down when it breaches the 100,000 item limit. Under these circumstances the purge queue then quickly grows and we have to empty the queue.

We noticed that when this happens if we attempt to empty the queue via the UI we quite often end up with our Database struggling with the mammoth task of performing a delete across hundreds of thousands of rows. We have instead resorted to running MySQL commands directly on the database to truncate the table.

Steps to reproduce

Configure a database queue and fill it with more than 100,000 records, preferably in excess of 500,000 to see the real performance hit of a delete statement. The AJAX will often time out and the database can struggle under severe load.

Proposed resolution

It seems like this could be done with a one-line change to the DatabaseQueue::deleteQueue() function and switch to use Connection::truncate() rather than Connection::delete(). A truncate is far more performant on large datasets as there are no row-level locks required to do so. Given that there are no arguments or conditions used when deleting the items in the queue, it seems safe to switch to a truncate.

Remaining tasks

User interface changes

None

API changes

None

Data model changes

None

CommentFileSizeAuthor
#2 purge-truncate_not_delete-3178028-2.patch508 bytesowenbush

Issue fork purge-3178028

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

owenbush created an issue. See original summary.

owenbush’s picture

Status: Active » Needs review
StatusFileSize
new508 bytes

Attached is a patch to use truncate rather than delete.

japerry made their first commit to this issue’s fork.

japerry’s picture

Status: Needs review » Fixed

  • japerry committed fe379b19 on 8.x-3.x
    Issue #3178028: Use Connection::truncate() instead of delete() when...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.