Problem/Motivation

While reviewing this module for suitability for a client I noticed that the queuing system appears to keep a global counter using Drupal's variable API (variable_set). Whenever a variable_set call is made, the variables cache is discarded and must be rebuilt on the next request received by Drupal. In high traffic, this can lead to many requests waiting on locks for the variables cache. I've seen it be the root cause of an entire system seizure.

Proposed resolution

The variables API was really only designed to be a store for config administration, not counters, locks, timestamps or tokens. Ideally, variables stored in the variables table should seldom change. If the variable being used is something that a computer can derive and maintain (such as the counter) then it should stored with a different system such as the cache system. While the cache system is not guaranteed to be persistent, using it in this manner won't break the variables cache and losing this information won't significantly impact the overall function of the site.

Comments

nielsvm’s picture

nielsvm commented
Status: Active » Closed (fixed)

Josh,

Thank you for your ticket and explanation. I entirely agree with you on how bad it is and that throwing away the variable cache for a simple counter is very dangerous, so I'm happy you've seen that.

See my commit which will be part of release 7.x-1.0:

(7.x-1.x *)$ git commit acquia_purge.install acquia_purge.module
[7.x-1.x b61c751] 2205015: by introducing _acquia_purge_state_set(), _acquia_purge_state_get(), _acquia_purge_state_initialize() and _acquia_purge_state_store_commit() I circumvented the need of storing state data to the variable system, which was dangerous as outlined in https://www.drupal.org/node/2205015. Now only one cache_set() will happen per page request that called _acquia_purge_state_set(), so its limited to the least possible amount.
 2 files changed, 134 insertions(+), 11 deletions(-)

I tested it as best as I could and everything seems to work just fine, closing the ticket as it'll be released not far from now.