Using Memcache API with AWS (Amazon) Elasticache

This example is from the installation docs:

$conf['memcache_servers'] = array(
  '10.1.1.1:11211' => 'default',
  '10.1.1.1:11212' => 'pages'
);
$conf['memcache_bins'] = array(
  'cache_page' => 'pages',
);

Is that what you mean? I'd be interested in hearing about using ElastiCache instead of a local Memcache instance.

It looks like you should be able to manage everything through the AWS docs - http://docs.aws.amazon.com/AmazonElastiCache/latest/GettingStartedGuide/...

Once the instances are created, just use those IPs in the Drupal configuration.

Making it more clear and understandable.

Need some advice for using this module with AWS ElastiCache.

If I have 1 clache cluster with multiple cache nodes, is it better to use each memcache "node endpoint" in the memcache_server setting or have only one memcache_server using the clusters "configuration endpoint"

Both methods appear to work, but im not sure which is the best/recommended approach.

any tips?

Ioze, you wrote:

"If I have 1 clache cluster with multiple cache nodes, is it better to use each memcache "node endpoint" in the memcache_server setting or have only one memcache_server using the clusters "configuration endpoint""

I have the same question. Did you configure your settings.php to use the node endpoints or the configuration endpoints? Is the config endpoint like some kind of load balancer (hoping...).

To answer my question, you should use the node endpoints. Here is what my config looks like:

$is_prod = TRUE;

if (file_exists('./'.conf_path() .'/settings.dev.php')) {
  include_once('./'.conf_path() .'/settings.dev.php');

  $is_prod = FALSE;

}

$conf['app_url'] = $_SERVER['APP_URL'];
$cookie_domain = $_SERVER['COOKIE_DOMAIN'];
$conf['awssdk2_access_key'] = $_SERVER['AWS_ACCESS_KEY_ID'];
$conf['awssdk2_secret_key'] = $_SERVER['AWS_SECRET_KEY'];

$databases['default']['default'] = array(
  'driver' => 'mysql',
  'database' => $_SERVER['DATABASE'],
  'username' => $_SERVER['DB_USERNAME'],
  'password' => $_SERVER['DB_PASSWORD'],
  'host' => $_SERVER['DB_HOST'],
  'prefix' => '',
);


if ($is_prod) {


  $databases['default']['slave'][] = array(
    'driver' => 'mysql',
    'database' => $_SERVER['DATABASE'],
    'username' => $_SERVER['DB_USERNAME'],
    'password' => $_SERVER['DB_PASSWORD'],
    'host' => 'myrdsreadsdb.eu-west-1.rds.amazonaws.com',
    'prefix' => '',
  );


  $conf['memcache_servers'] = array(
    'ecnode.0001.euw1.cache.amazonaws.com:11211' => 'default',
    'ecnode.0002.euw1.cache.amazonaws.com:11211' => 'entity',
    'ecnode.0003.euw1.cache.amazonaws.com:11211' => 'session',
    'ecnode.0004.euw1.cache.amazonaws.com:11211' => 'users',
  );
  $conf['memcache_bins'] = array(

    'session' => 'session',

    'users' => 'users',

    'cache' => 'default',
    'cache_bootstrap' => 'default',
    'cache_field' => 'default',
    'cache_filter' => 'default',
    'cache_image' => 'default',
    'cache_libraries' => 'default',
    'cache_menu' => 'default',
    'cache_page' => 'default',
    'cache_path' => 'default',
    'cache_rules' => 'default',
    'cache_token' => 'default',
    'cache_update' => 'default',
    'cache_variable' => 'default',
    'cache_views' => 'default',
    'cache_views_data' => 'default',

    // cache entity
    'cache_entity_file' => 'entity',
    'cache_entity_message' => 'entity',
    'cache_entity_message_type' => 'entity',
    'cache_entity_message_type_category' => 'entity',
    'cache_entity_node' => 'entity',
    'cache_entity_taxonomy_term' => 'entity',
    'cache_entity_taxonomy_vocabulary' => 'entity',
    'cache_entity_user' => 'entity',
  );


  $conf['session_inc'] = 'sites/all/modules/memcache/unstable/memcache-session.inc';
  $conf['cache_backends'][] = 'sites/all/modules/memcache/memcache.inc';
  $conf['cache_default_class'] = 'MemCacheDrupal';
  $conf['memcache_key_prefix'] = 'mc1';
  $conf['page_cache_without_database'] = TRUE;
  $conf['page_cache_invoke_hooks'] = FALSE;
  $conf['cache_class_cache_form'] = 'DrupalDatabaseCache';

  $conf['lock_inc'] = 'sites/all/modules/memcache/memcache-lock.inc';
  $conf['memcache_stampede_protection'] = TRUE;

}

Hello,

I am using Elastic cache with Drupal with Memcache API , but getting strange issue i.e when ever i update some content in Drupal and save it after some time it gets reverted , can u let me know what i am doing wrong?

without elastic cache my site runs fine.

my setting.php looks like below for with memcahe settings:

$conf['cache_backends'][] = 'sites/all/modules/contrib/memcache/memcache.inc';
$conf['cache_default_class'] = 'MemCacheDrupal';
$conf['cache_class_cache_form'] = 'DrupalDatabaseCache';
$conf['page_cache_without_database'] = TRUE;
$conf['page_cache_invoke_hooks'] = FALSE;
#$conf['lock_inc'] = 'sites/all/modules/memcache/memcache-lock.inc';
$conf['memcache_servers'] = array('xxx.xx.xxx.xxx.cache.amazonaws.com:11211' => 'default');
$conf['memcache_key_prefix'] = 'xxx';

$conf['lock_inc'] = 'sites/all/modules/contrib/memcache/memcache-lock.inc';
$conf['memcache_stampede_protection'] = TRUE;

i tried with and without stampede_protection and 'lock_inc variables but do not get fix issue,

any help will be appreciated

Thanks

I've seen the same issue that Yogen has reported for two different clients now (different companies, both using Elasticache). Same "workaround" i.e. don't use Elasticache!

Not got to the bottom of it yet, but would appreciate any ideas.

Hi,

I'm having issues with Elasticache, but its very intermitten. Can the OP or someone updated me on the status of this issue, and wether or not its still discouraged to use this module with AWS Elasticache?

So am I correct to understand from the recent posts that 'Memcache API and Integration' does not work properly with AWS Elasticache?

We enabled Memcache settings in late December and are seeing data change state on pages when reloading (even node/edit screens, and we have cache_form hitting database!) You see three different values in a field's form input as you refresh the page repeatedly.

Clearing all cache temporarily corrects the issue, but that's not a workable remedy.

Should we be using 'Memcache Storage' with Elasticache, instead?

For anyone who may read this, here are our settings ... memcache is storing and active (and our database activity level went from 60%, and crashing a lot, to under 10%), but apparently the flushing and setting of cache is alternating between different servers. Our developers thought maybe there was an errant node or field hook interfering with normal operation, but now I'm beginning to wonder if I've chosen the wrong module for matching D7 to Elasticache.

Like others in this thread, would also love some advice, here. Like, would explicitly listing the Elasticache cluster addresses, in 'memcache_servers' setting, have any benefit? Thx.

$conf['cache_backends'][] = 'sites/all/modules/contrib/memcache/memcache.inc';
$conf['cache_default_class'] = 'MemCacheDrupal';
$conf['cache_class_cache_form'] = 'DrupalDatabaseCache';
$conf['cache_class_cache_update'] = 'DrupalDatabaseCache';
$conf['memcache_stampede_protection'] = TRUE;
$conf['lock_inc'] = 'sites/all/modules/contrib/memcache/memcache-lock.inc';

I've added some documentation to the README.txt regarding how to use the Memcache module with Elasticache. It's always been supported and works, but if you want to use Automatic Discovery you need to use Amazon's fork of the PECL Memcached extension:

http://cgit.drupalcode.org/memcache/commit/?id=9b6bad8

Thanks, Jeremy. I appreciate that.

We did it get it working, and the issue was something undocumented with Amazon

In our case, our configuration was correct, and the correct PHP libraries were all in place.

The problem was the cluster address alias used in Auto Discovery (or maybe it was the Cluster Client) library needs to have ".cfg." in the name and ours did not.

Our Ops provider renamed the alias and rebuilt the library in PHP and everything works great!

We were driving ourselves crazy for a few weeks before someone finally figured it out.

So, to anyone else who reads this, it definitely works and you DON'T have to point to a single node endpoint, as some have done in the past.

:)

Here is the AWS documentation for the Configuration Endpoint. As you can see, the example uses ".cfg." but does not explicitly say you have to name yours as such. Perhaps Amazon thought it was implicit in the instructions, but our Ops team did not think so.

http://docs.aws.amazon.com/AmazonElastiCache/latest/UserGuide/Endpoints....

Thanks; I updated the documentation to mention this, and to include the link to AWS documentation.