Hi,

we are using AdvAgg on our busy Drupal site, and are having deadlocks in the database as AdvAgg seems to systematically set a lock named "locale_js_alter".

Usually, all the CSS/JS files have already been generated and ready to download. It should only leave the inline JS (Drupal.settings) to compress.
I tried to disable compressing inline CSS and JS, but we still get the locks.

Currently, in nearly all of our requests, the only writes on the database are caused by the insertion and deletion of the semaphores (7M per day). Sometimes we experience deadlocks on the semaphore table.

Is there any way to prevent the creation of the lock, especially when there is nothing to do ?
Did I miss something ?

Eventually, delegating locks into Memcache could be a viable solution ?

Thanks
Franck

CommentFileSizeAuthor
#9 advagg-2798013-7-only-acquire-lock-if-needed.patch2.09 KBmikeytown2
#5 advagg-2798013-5-only-acquire-lock-if-needed.patch2.09 KBmikeytown2
#4 advagg-2798013-4-only-acquire-lock-if-needed.patch1.81 KBmikeytown2
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

franckmarquis created an issue. See original summary.

mikeytown2’s picture

mikeytown2 CreditAttribution: mikeytown2 commented

This issue explains the why #2469063: locale_js_alter() - PDOException: Integrity constraint violation: 1062 Duplicate entry 'javascript_parsed'

Long story short locale_js_alter() uses a bad design pattern (abuse of variable_set) so I put a lock around it so only one instance of it can run at a time. variable_set() is usually more expensive than lock_acquire() and locks can be offloaded to things like memcache as you mentioned.

If show engine innodb status says that the deadlocks are on the semaphore table, that's ok. This table is designed to have things like that happen to it. When analyzing the output of deadlocks always check the date.

Also checkout https://www.drupal.org/project/apdqc It's fixed all the DB cache and lock issues I used to have.

mikeytown2’s picture

mikeytown2 CreditAttribution: mikeytown2 commented

Looking at my code, I think I can do some checks before acquiring the lock.

mikeytown2’s picture

mikeytown2 CreditAttribution: mikeytown2 commented
Status: Active » Needs review
FileSize
advagg-2798013-4-only-acquire-lock-if-needed.patch1.81 KB
mikeytown2’s picture

mikeytown2 CreditAttribution: mikeytown2 commented
FileSize
advagg-2798013-5-only-acquire-lock-if-needed.patch2.09 KB

Found another condition where it might need to be checked.

Status: Needs review » Needs work

The last submitted patch, 5: advagg-2798013-5-only-acquire-lock-if-needed.patch, failed testing.

The last submitted patch, 5: advagg-2798013-5-only-acquire-lock-if-needed.patch, failed testing.

The last submitted patch, 5: advagg-2798013-5-only-acquire-lock-if-needed.patch, failed testing.

mikeytown2’s picture

mikeytown2 CreditAttribution: mikeytown2 commented
Status: Needs work » Needs review
FileSize
advagg-2798013-7-only-acquire-lock-if-needed.patch2.09 KB

Fixes the typo

franckmarquis’s picture

franckmarquis CreditAttribution: franckmarquis commented

Wow, that was fast...
First tests seem fine, I just need to do more extensive checks but I will only be able to do them on monday or tuesday

Great work, great service, and thanks for the explaination !

Our only deadlocks are on the semaphore table, like you said. It may be normal, but it still generate slowdowns on our site when it happens. This patch should be a nice optimization on high trafic sites, I think.

I already have seen your apdqc module while searching for optimization techniques, but as our cache is on memcache and not in mysql, I thought it was not needed.

Thanks again !

  • mikeytown2 committed e8517b5 on 7.x-2.x 
    Issue #2798013 by mikeytown2: AdvAgg systematically creates locks, only...
mikeytown2’s picture

mikeytown2 CreditAttribution: mikeytown2 commented
Status: Needs review » Fixed

Should be good. Committed #9

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.