My setup is:

1 load balancer (haproxy)
2 webservers (server01/server02 - each running varnish and memcached)
1 NFS (both webservers share files)
1 MySQL server

I have set up memcache in my settings.php file with the following config:

        $conf['memcache_options'] = array(
          Memcached::OPT_DISTRIBUTION => Memcached::DISTRIBUTION_CONSISTENT,
        );

        $conf['cache_backends'][] = 'sites/all/modules/memcache/memcache.inc';
        $conf['lock_inc'] = 'sites/all/modules/memcache/memcache-lock.inc';
        $conf['memcache_stampede_protection'] = FALSE;
        $conf['cache_default_class'] = 'MemCacheDrupal';
        $conf['cache_class_cache_form'] = 'DrupalDatabaseCache';
        $conf['page_cache_without_database'] = TRUE;
        $conf['page_cache_invoke_hooks'] = FALSE;

        $conf['memcache_servers'] = array('server01:11211' => 'default', 'server02:11211' => 'default');
        $conf['memcache_bins'] = array('cache' => 'default');

        $conf['memcache_log_data_pieces'] = 10;

This setup runs fine until one of the servers is taken down. The remaining server seems to hang and the load balancer reports all servers down. I can temporarily fix this issue by changing the memcache_servers variable to localhost:11211. Obviously this temporary setup will not work with both servers up as the memcache becomes out-of-sync. I have tried with memcache_stampede_protection both true and false, both setups have the same issue.

Drupal core - 7.50
Memcache module - 7.x-1.5+4-dev

Has anyone had the same issue and found a solution?

Thanks

Tom

Comments

tombsage created an issue. See original summary.

jgrubb’s picture

jgrubb commented

Hey Tom, this is due to the design of memcached. Memcached is a distributed object store, and the logic for which member of the pool contains an object is wrapped up in the client. If a member of the pool goes down, the pool doesn't automatically rebalance itself, that member has to be removed from the pool (like when you set it to localhost).

There is no solution to this on the Drupal end of things. For what it's worth this is a known issue and will result in exactly the behavior you're describing.

jgrubb’s picture

jgrubb commented
Status: Active » Closed (works as designed)