I have a site that has a large traffic, I use 3 memcache instance but the synchronization between the 3 does not work well with drupal.
thank you for helping me to configure the 3 server memcache or to indicate me debugging tools.
I use this module for drupal.
Thanks.

Issue fork memcache-3077632

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

mh.marouan created an issue. See original summary.

fgm’s picture

Note that memcached does not perform replication, so there is no synchronization involved between your memcached servers.

On the contrary, memcache is optimized to use sharding, and this is what the module documentation describes:

  • "clusters" are groups of memcached instances
  • "servers" are these memcached instances
  • "bins" are the basic cache services in Drupal, implemented as tables in the default database cache implementation

With the memcache plugin, you define which bins go into which clusters, and which servers belong to which bin.

You must take care not to add a single server in multiple clusters, and also make sure that the definitions of the servers are identical on all web servers if you use multiple web servers, assuming you're not using 127.0.0.1 or a unix socket to link to a memcache serveur, because these would not be the same on each web server.

All of this is explained in detail in the module documentation.

jeremy’s picture

Multi-server configuration is described in more detail here:
https://git.drupalcode.org/project/memcache/blob/7.x-1.x/README.txt#L73

And here's an example similar to what you describe:
https://git.drupalcode.org/project/memcache/blob/7.x-1.x/README.txt#L268

khaled.zaidan’s picture

I think I'm having similar issues to this.

In my setup, I have a single cluster with two servers. I've found that whenever anything is written to memcache, it's only written on one of the two servers. This means that the other server immediately has stale data that could be loaded in a subsequent request.

I've read and re-read the README file trying to see if I missed anything (and I'm actually still not 100% sure as there are parts I don't fully understand*!).

Is this working as designed? Is there a way to have the any write operation apply to all servers so that any read is guaranteed to get fresh data?

Any insight on this is appreciated.

*I mainly didn't get what the memcache distribution is what setting it to consistent does.

khaled.zaidan’s picture

I used the following code snippet to test the issue in my previous comment (I ran the command with drush, hence the echos).

At first I would see the value come back from one of the memcache servers and no result from the other. Then, I started seeing one of the boxes returning the latest value while the other returns an old one.

use \Drupal\memcache\MemcacheSettings;
use \Drupal\Core\Site\Settings;
use \Drupal\memcache\Driver\MemcacheDriverFactory;

function test_memcache() {
  $time = time();
  echo 'Value is: ' . $time . "\n\n";
  \Drupal::cache()->set('test_memcache_x', $time);

  // Get our memcache settings.
  $memcache_settings = \Drupal\Core\Site\Settings::get('memcache', []);

  // Create a partial settings array.
  // We'll isolate each memcache server here and use it to create
  // a MemcacheDriverFactory.
  $pseudo_settings_array = ['memcache' => $memcache_settings];

  // Loop over all our servers.
  foreach ($memcache_settings['servers'] as $server => $bins) {
    // These 3 lines are building the MemcacheSettings object.
    $pseudo_settings_array['memcache']['servers'] = [$server => $bins];
    $pseudo_settings_object = new Settings($pseudo_settings_array);
    $memcache_settings_object = new MemcacheSettings($pseudo_settings_object);

    // Create the MemcacheDriverFactory.
    $memcache_factory = new MemcacheDriverFactory($memcache_settings_object);

    // Read the value from this memcache server.
    $value = $memcache_factory->get('default')->get('test_memcache_x');
    echo $server . ': ' . $value->data . "\n";
  }
}

// Memcache server/cluster settings in settings.php.
$settings['memcache']['servers'] = [
  'memcache01:11211' => 'default',
  'memcache02:11211' => 'default',
];
fgm’s picture

By default, as I described in #2, the standard Memcached daemon does not provide replication between instances, and by default the drivers don't force it at the client level (hard and costly because of CAP issues).

However, you could use an implementation providing replication, like Hazelcast with memcached protocol, or repcached or the replication option in twemproxy or mcrouter.

Or just like our high traffic sites, configure default memcache with memcache extension (not memcached), ensure consistent hashing, and double check your configuration so that whichever front-end is hit, the same key always goes to the same memcached instance.

o'briat’s picture

Hi,

I'm using the memcached php module, this client should provide server failure (https://www.php.net/manual/en/memcached.constants.php#memcached.constant...), so if a server is no more available the sharding method is updated and missing keys (previously allocated to the missing server) should be recreated on servers still available.
The performance loss should be punctual and inferior to a single drush cr.

But it doesn't seems to be the case: I tested it, I shutdowned one server (on two) and all performances were falling down. Everything returned to normal once the server was back online.

Am I missing something?

Also I don't get the point of using multiple bins and allocated them on specific memcached server/cluster. It looks like micro optimization, the eventual use cases should be better documented.

morla’s picture

i am having the same isssue @o'briat is describing.

japerry made their first commit to this issue’s fork.

japerry’s picture

Status: Active » Closed (works as designed)

Sharding is by design, added documentation plus tests that demonstrate it, and the failover half from comment 7/8 is real but belongs to #3462456 — where the finding is that OPT_REMOVE_FAILED_SERVERS already works today via $settings['memcache']['options'].

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.

o'briat’s picture

@japerry
That's great!
But I don't fully understand your reply, if OPT_REMOVE_FAILED_SERVERS is working, so there's should be no performance problems and issue #3462456 could be closed, no?