Due to #1679344: Race condition in node_save() when not using DB for cache_field, Drupal.org needed to move the field cache to the DB. Our DB servers have been running into deadlocks which occasionally take the site down for up to ~5 minutes at time, and sometimes leads to replication breaking.
The current recommendation for Drupal.org is https://www.drupal.org/project/cache_consistent with Method #1 (alternate)
Our DB isolation level is read-committed, so method #1 is the one to use. Fabianx reviewed the code and liked that ConsistentCacheLookup
had the buffer in its own class.
Deployment
Done in BZR:
- Upgrade memcache from 7.x-1.0+5-dev to 7.x-1.3-rc1
- Add cache_consistent 7.x-1.1
- Apply core patch
Update settings.local.php and test on git7staging.
Deploy to production, enable cache_consistent, and update settings.local.php on both util & www*.
Comments
Comment #1
drummRudy and I tentatively plan to deploy this around 11am US Pacific time tomorrow, in 22 hours. Downtime is not expected for deployment, but is possible if we need to roll back.
Comment #2
drummComment #3
drummI went ahead and reviewed the latest updates to the memcache module. They look good and straightforward. The new stampede protection looks like something that will help us as well.
I've committed all the code to BZR, since git7site and stagingwww are our only real options for testing, and we don't have other upcoming deployments.
Comment #4
drummThis is deployed to git7site, and passing BDD tests well enough. (The test failures are related to not having a staging cgit setup, as usual.)
Comment #5
drummComment #6
drummWe're moving forward with this deployment.
Comment #7
drummDeployed!
Comment #8
drummComment #9
gielfeldt CreditAttribution: gielfeldt commentedSorry to post in a closed ticket, but I'm just curious to know if everything went as planned, or if you experienced any problems.
Comment #10
drummSo far it has been good. I haven't heard about any symptoms of cache inconsistency. What we had before was:
node_save()
that adds data to file fields.We've still been avoiding cache clears, the initial cache filling was okay.