According to cache_set() a cache entry could be created with a fixed expiry date by passing a unix timestamp as fourth parameter. But this the invalidation of these entires is currently not handled at all and memcache delivers outdated cache entries.

CommentFileSizeAuthor
#2 memcache_storage-fixed_expiry_date-2096935-1.patch1.47 KBmkalkbrenner

Comments

spleshka’s picture

spleshka
he/him
Primary language English
Location UAE
commented
Status: Active » Closed (works as designed)

Hi Markus,

Did you looked into the MS code? :) Please, see this, and pay your attension at this lines:

// Otherwise simply convert default drupal expire time to memcache expire time.
elseif ($expire > REQUEST_TIME) {
 $expire -= REQUEST_TIME;
}

So the timestamp is converted to the memcached TTL format.

By the way, if timestamp is passed without substruct of REQUEST_TIME, memcached will consider it as a unix timestamp, and item will be expired properly anyway.

mkalkbrenner’s picture

mkalkbrenner
Primary language German
Location 🇩🇪
commented
Status: Closed (works as designed) » Needs review
StatusFileSize
new memcache_storage-fixed_expiry_date-2096935-1.patch1.47 KB

Created a patch to solve this issue.

spleshka’s picture

spleshka
he/him
Primary language English
Location UAE
commented
Status: Needs review » Closed (works as designed)

Oh, I understand what do you mean. But memcached will not fetch expired items from its pool. So no need to check already expired items in validation, because if current timestamp is greater then cache item expiration timestamp, memcached will return FALSE instead of item.

spleshka’s picture

spleshka
he/him
Primary language English
Location UAE
commented

If you dont trust me, then just execute following code:

cache_set('cid', 'test', 'cache', REQUST_TIME + 3); // Cache for 3 seconds.
sleep(3); // Wait for 3 seconds.
$cache = cache_get('cid', 'cache'); // Try to fetch expired item.
dpm($cache); // Should be FALSE here.
mkalkbrenner’s picture

mkalkbrenner
Primary language German
Location 🇩🇪
commented

During posting #2 I was not aware of that you already posted comment #1 => Race Condition ;-)

Your snippet works.

spleshka’s picture

spleshka
he/him
Primary language English
Location UAE
commented

So no more problems with cache invalidation? Please, feel free to send me any feedbacks about Memcache Storage.

mkalkbrenner’s picture

mkalkbrenner
Primary language German
Location 🇩🇪
commented

Indeed, we have some strange problems. Basically it seems that the invalidation is not 100% reliable:

  • If we rollout new code / modules and do drush updb; drush cc all we sometimes face some "strange behaviors" of the site until we restart the memcached.
  • In the apachesolr module the index status page doesn't update correctly.
  • I assume that this issue is somehow related: #2060277: menu links translations randomly disappear
  • ...

That's why I started to search through your module to find a case where a cache entry is not invalidated correctly.

spleshka’s picture

spleshka
he/him
Primary language English
Location UAE
commented

Oh.. Are there any possibility to debug this together? I want to find the problem, because till your issue I never heard about problems with expiration.

mkalkbrenner’s picture

mkalkbrenner
Primary language German
Location 🇩🇪
commented

I'm just talking to my colleagues how we could reproduce the errors "reliable" to have the chance to debug it.

Meanwhile some details about our setup. We're running drupal multisite setup deployed on multiple servers. memcached runs on two servers. This is one of the configurations:

$conf['cache_backends'][] = 'sites/all/modules/contrib/memcache_storage/memcache_storage.inc';
$conf['cache_default_class'] = 'MemcacheStorage';
$conf['cache_class_cache_form'] = 'DrupalDatabaseCache';
$conf['memcache_servers'] = array(
  '192.168.1.1:11211' => 'default',
  '192.168.1.12:11211' => 'default',
);
$conf['memcache_storage_key_prefix'] = 'site_1'; // every site uses it's own prefix
$conf['memcache_extension'] = 'Memcached';
$conf['memcache_options'] = array(
  Memcached::OPT_COMPRESSION => FALSE,
  Memcached::OPT_DISTRIBUTION => Memcached::DISTRIBUTION_CONSISTENT,
  Memcached::OPT_TCP_NODELAY => TRUE,
  Memcached::OPT_BINARY_PROTOCOL => TRUE,
 );
$conf['lock_inc'] = 'sites/all/modules/contrib/memcache_storage/includes/lock.inc';
spleshka’s picture

spleshka
he/him
Primary language English
Location UAE
commented

Could you temporary disable one of the memcached servers? I mean leave only one server and try to run drush cc all again and see if problem remain. May be problem in network communication.

spleshka’s picture

spleshka
he/him
Primary language English
Location UAE
commented

By the way, for you set up is better to enable compression, because you use tcp instead of sockets, so you loose some time on the network delivery. And if the data will be smaller - the delivery should be faster.

mkalkbrenner’s picture

mkalkbrenner
Primary language German
Location 🇩🇪
commented
Title: Cache entries with fixed expiry date are not deleted » Sometimes the invalidation of cache entries is not reliable
Category: bug » support
Priority: Major » Normal
Status: Closed (works as designed) » Active

Thanks for your hint regarding the compression.

We'll do some intensive tests including the shutdown of one memcached next week during low traffic hours.

Meanwhile we try to reproduce the various errors in the development and the staging environment. I'll keep you informed ...

spleshka’s picture

spleshka
he/him
Primary language English
Location UAE
commented

Great, looking forward for the tests results.

mkalkbrenner’s picture

mkalkbrenner
Primary language German
Location 🇩🇪
commented

We did a *lot* of tests and tuned our memcached installation. But currently we're only running one memcached server. So far we're not able to reproduce bugs, but we'll wait another week if an error occurs.

After that, we'll probably re-enable the memcache distribution on multiple servers and do some more tests.

One thing we noticed is the way the drupal core update module works. It's possible to access the "cache_update" using cache_get, cache_set and cache_clear_all, but internally the module uses hard coded sql to access the cache_update table. Therefore we added $conf['cache_class_cache_update'] = 'DrupalDatabaseCache'; to avoid possible bugs.

spleshka’s picture

spleshka
he/him
Primary language English
Location UAE
commented
Assigned: Unassigned » spleshka
Status: Active » Needs work

Thanks a lot for a tests! So I see that currently there are no issues with cache invalidation. But for cache_update I think it would be nice to have a couple notes in the README about such hardcoded behavior.

spleshka’s picture

spleshka
he/him
Primary language English
Location UAE
commented
Issue summary: View changes

fixed typo

spleshka’s picture

spleshka
he/him
Primary language English
Location UAE
commented
Issue summary: View changes
Status: Needs work » Fixed

I've commited and pushed the data about cache_update in the README. I think that issue is solved for now, am I right?

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.