Support for multiple redis Instances

Good idea supporting multiple Redis instances but this won't be for the first real stable release I am preparing right now. Postponing until the stable release comes out. This issue is still open for discussion.

Currently the 2 implementations of redis phpredis and predis use the "keys" command which is not supported in redis clustered implementations.

Something like this could be used to grab keys from all the instances. Then just change all the lines in the clear function from "$client->keys" to "$this->keys"

  	function keys($k){
  		global $conf;
  		if(is_array($conf['redis_client'])){
  			$all_keys = array();
  			foreach($conf['redis_client'] as $r){
  				$client = new Predis\Client(array('host' => $r[0], 'port' => $r[1]));
				$all_keys = array_merge($client->keys($k), $all_keys);
			}		
			return $all_keys;
  		}else{
  			$client = Redis_Client::getClient();
  			return $client->keys($k);
		}
	}

Also using Predis, the library does not support high availability. If one redis server is down the whole thing breaks. We should implement a system for connecting to the remaining redis servers if one is down.

As soon as you want to do sharding or something similar, you loose a lot of commands, not only KEYS but WATCH/MULTI/EXEC blocks as well. This module, as it exists, can *not* support this. The only multi-server installation it can support is basic master/slave replication. Supporting any other configuration would mean loosing transactions. As long as you use it for cache, and provide locking throught a single server, it might be OK, but the cache backend implementation the module provides would need a serious refactor. EDIT: Actually the cache backend uses MULTI blocks only for pipelining so I guess it would be easier than I think.

Another thing to consider is that it's might be easier to use a proxy such as twemproxy for high-availibility instead of relying on PHP code to do this: PHP code would remain minimal and easy to maintain for the cost of having to install such proxy.

@pounard: does this module work with twemproxy? I have twemproxy running and can set/get keys via redis-cli, but setting Redis configuration in settings.php to point at the twemproxy port returns RedisException: read error on connection in Redis->watch().

This is probably because of the WATCH command, twemproxy doesn't support transactions. I never tested it myself using it, may be I should and try to make it run using it! Beware that it is impossible to run the lock backend because locking relies on transactions, are you using the cache backend only or the lock inc as well?

Hmm, I followed the README and other guides found for configuring Redis module for Drupal. All seem to set both, so this is what I have been using:

$conf['lock_inc'] = 'sites/all/modules/redis/redis.lock.inc';
$conf['cache_backends'][] = 'sites/all/modules/redis/redis.autoload.inc';

At this point, are all implementations of Redis with Drupal currently only using a single Redis cache instance?

At this point you are using the lock backend but not the cache backend, read the readme file again, you need to set the default cache backend class:

$conf['cache_default_class']    = 'Redis_Cache';

And comment the following line before retrying:

// $conf['lock_inc'] = 'sites/all/modules/redis/redis.lock.inc';

It should then work I guess.

Oh, I have all that other configuration, I just didn't include it. Can you explain what the lock backend does and why it can't be used in this situation?

Locking needs write operations to be atomic in order to ensure that one and only one concurrent thread can get the same lock at the same time, for this I used the MULTI/WATCH/EXEC Redis blocks which ensure atomicity of the transaction when correctly used. Transactions cannot be ensured in an environment where you dispatch information into multiple servers, this is due to sharding and partionning, you cannot ensure atomicity of a single transaction using keys accross multiple different servers, that's why twemproxy don't support transactions.

Ok, this partially works with the lock backend commented out. Some issues:

1. Cache clears from the UI or from Drush fail due to the KEYS command not being supported by twemproxy:
   

   RedisException: read error on connection in Redis->keys() (line 161 of /sandbox/d7/sites/all/modules/contrib/redis/lib/Redis/Cache/PhpRedis.php).
   

2. "Failover" configurations described in #1824146 don't work since twemproxy doesn't yet support the PING command. We'd like to use this so things degrade to MySQL if Redis happens to be down, so users aren't delivered PHP errors.

Not sure there's anything we can do here until these commands are added to twemproxy. We will likely just run a single dedicated Redis instance for now. Thoughts?

As I said, I think that it will need a major rewrite to be able to work using multiple instances, especially when doing sharding.

But you can still use master-slave replication if you really need redundancy, I'd advise you to read this: http://redis.io/topics/replication

The problem in our case (cache usage) will always be the cache clear operations: if we want to be able to drop everything from a cache bin, we need to register its keys somewhere to be able to send the clear command. This where the problem starts because as soon as we try to register all keys somewhere, we always end up with a non scalable storage and/or algorithm for clearing the caches, or completly change the way we store keys and maybe use set or hashes instead, but that'd need a complete rewrite and would probably make sharding inneficient.

@vinmassaro i was able to hack this module into working with multiple redis instances running by using Predis instead of twemproxy and extending the predis classes to create my own redis backend for the module. It's really easy to do, just override the keys methods and any other methods that aren't supported by redis cluster to send those commands individually to each node and return a merged array of keys. Predis supports clustering out of the box, the only thing it's missing is the commands that are not a part of the redis cluster specification... just fix that

@richwandell: thanks for the info. We ended up going with a master Redis instance and a replicated slave with failover via Redis Sentinel. This seems to be sufficient for our needs as an LRU cache store.

@richwandell
Did you have any success with hacking? If 'yes', could you share your solution?

What's the status on this? What's the maintainer's stance on commiting this if accomplished?
I see the discussion going the master/slave sharding path, but multiple Redis backends would also allow spearation of individual cache bins into separate Redis instances alas MongoDB cache backend allows (eg. one with disk persistance and another without, depending on warmup times).
Depending on number of items and storage availability, it might be also a good fit for a shared and truly expirable 'cache_form' bucket and solve some old known 'cache_form' headaches for some setups.
Setting this back to active to reopen discussion and attaching to 7.x-2.12 (could not found 7.x-2.x).

To be honest, I never wanted to implement this feature, but I might give it a try in the 3.x release, anyway, I'm still leaving this opened since a lot of people seem to be interested, it makes me think I should do it.

I will try, when I'll have the time to do it, may be in a few weeks.

Hello guys, I make some patch for cluster support.

It works with sharding type of cluster.

Patch works only with Predis library, please check it.

It's very very beta patch :)

Sample of settings.php file:

$conf['redis_client_interface'] = 'Predis';
$conf['cache_backends'][] = 'sites/all/modules/contrib/redis/redis.autoload.inc';
$conf['cache_default_class'] = 'Redis_Cache';
$conf['cache_class_cache_form'] = 'DrupalDatabaseCache';
//$conf['lock_inc'] = 'sites/all/modules/contrib/redis/redis.lock.inc';
$conf['redis_backends'] = array(
  array(
    'host' => 'localhost',
    'port' => '6379',
    'database' => 1,
  ),
  array(
    'host' => 'localhost',
    'port' => '6380',
    'database' => 1,
  )
);
//$conf['redis_client_host'] = 'localhost';
//$conf['redis_client_port'] = '6379';
//$conf['redis_client_base'] = 1;
$conf['redis_flush_mode'] = 1;
$conf['redis_perm_ttl'] = '1 month';
$conf['cache_prefix'] = 'drupal_cache-test';

I am actually working on multi-server configuration :)

This definitely won't be a feature of the 2.x branch.

As a side note: if your need is to have a distributed system, 3.x version is able to use a shard through proxy assisted sharding, and connecting to a redis-cluster is a planned feature, see #2556097: Support cluster connections.

I'm closing this issue now, because I won't implement it ever in the 3.0 release. 4.0 may bring it someday.

Setting to 'postponed' instead, to help keep track of the request, as done for #2556097: Support cluster connections.
It would be a pity to lose this thread unnoticed when the time comes.

Fair enough.

I am sorry, this will not get into 7.x-3.x, I am working on a new PHP library outside of Drupal: https://github.com/makinacorpus/redis-bundle which provides a more fully-featured connector for this, which also provides the Drupal 7 backend. Future 7.x-4.x version will be this library integration.