Problem/Motivation

There are often issues reported(for example this one) of MySQL failing on multiple request trying to save an entity of the same type. This happens due to MySQL transaction deadlocks.

The "semaphore" table, which(I suppose) was meant to deal with this kind of problems, suffers from this issue as well - there is a bug report of it, but I was also able to reproduce it locally too.

An example of steps to reproduce

  1. Write a batch that generates dummy entities, for example Nodes
  2. Launch a few of them in separate browser tabs or windows
  3. At least one of the batches will fail quickly

Proposed resolution

Wrap the ContentEntityStorageBase::doSave() into GET_LOCK and RELEASE_LOCK functions of MySQL with using entity type specific lock ID's, for example "entity_node_save" or "entity_commerce_order_save".

The request

We(my company) are going to implement this solution as a separate module which would use hook_entity_presave/hook_entity_update, and publish it in case of success. We have a proof-of-concept code, which has shown us great stable results.

Can you guys please take a look onto this idea and tell if it is worth development and publishing as a contrib module for the community? Also, are there maybe any "underwater stones" to consider specifically?

CommentFileSizeAuthor
#10 2824097-test.patch21.68 KBranjith_kumar_k_u
#9 3216606-9.patch5.11 KBranjith_kumar_k_u
#8 3216606-nr-bot.txt107 bytesneeds-review-queue-bot
#6 locked_content_entity_operations.patch4.6 KBleonidzinets

Comments

leonidzinets created an issue. See original summary.

mxr576’s picture

mxr576
Primary language Hungarian
Location Hungary
commented
Version: 9.0.x-dev » 9.4.x-dev
Issue tags: +concurrency
cilefen’s picture

cilefen commented

I would rather we fix this in core. Is the semaphore locking fixable?

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

leonidzinets’s picture

leonidzinets
Primary language Ukrainian
commented
StatusFileSize
new locked_content_entity_operations.patch4.6 KB

After some extra research we found out that implementing locks in entity-related hooks is not enough, and under some circumstances the issues still happened. So, this is the approach that we've currently ended upon. It's a core file patch(attached) and so far initial testing has shown good results.

Do you guys know if there are some underwater stones with this approach? Were there some reasons why this(or similar) approach hasn't been implemented to the core yet? Maybe there are some extra consequences of this approach which I currently don't understand?

cilefen’s picture

cilefen commented
Title: [RFC] Usage of GET_LOCK/RELEASE_LOCK functions for multirequest thread safety » Usage of GET_LOCK/RELEASE_LOCK functions for multirequest thread safety
Category: Support request » Feature request
Status: Active » Needs review
Issue tags: -Entity system, -MySQL deadlock, -concurrency
needs-review-queue-bot’s picture

needs-review-queue-bot commented
Status: Needs review » Needs work
StatusFileSize
new 3216606-nr-bot.txt107 bytes

The Needs Review Queue Bot tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".

Apart from a re-roll or rebase, this issue may need more work to address feedback in the issue or MR comments. To progress an issue, incorporate this feedback as part of the process of updating the issue. This helps other contributors to know what is outstanding.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

ranjith_kumar_k_u’s picture

ranjith_kumar_k_u commented
StatusFileSize
new 3216606-9.patch5.11 KB

Re-rolled #6

ranjith_kumar_k_u’s picture

ranjith_kumar_k_u commented
StatusFileSize
new 2824097-test.patch21.68 KB

wrong patch please ignore

ranjith_kumar_k_u’s picture

ranjith_kumar_k_u commented

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.