Enabling a backup cache engine if first cache engine fails

Interesting idea, looks fine! So let's gather reviews from community and I glad to see this in upcoming release.

It seems like if you are going to implement a caching scheme like this, you might want to consider making the number of caching layers arbitrary. For example:

$conf['cacherouter'] = array(
  'default' => array(
    'engine' => 'memcache',
    'priority' => 1,
    'server' => array(''),
    'shared' => TRUE,
    'prefix' => '',
    'path' => '',
    'static' => FALSE,
    'fast_cache' => FALSE,
  ),
  'backup1' => array(
    'engine' => 'apc',
    'priority' => 2,
    'server' => array(''),
    'shared' => TRUE,
    'prefix' => '',
    'path' => '',
    'static' => FALSE,
    'fast_cache' => FALSE,
  ),
  'backup2' => array(
    'engine' => 'db',
    'priority' => 3,
    'server' => array(''),
    'shared' => TRUE,
    'prefix' => '',
    'path' => '',
    'static' => FALSE,
    'fast_cache' => FALSE,
  ),
);

@cweagans no it's different, priority has no meaning because 'backup1' and 'backup2' is cache tables not an abstract bins.

I don't know if you've seen the old issue http://drupal.org/node/365088 where something like this was discussed (and dropped?) earlier, seems like there are some old thoughts/concerns in there.

Anyway, I'd love to see this in CacheRouter; we've been running a similar system that would

- Try to fetch from cacherouter (apc)
- Success?
-- Yes: Return value
-- No: Try to fetch from cacherouter (database)
-- Success?
--- Yes: Store in APC, return value
--- No: Generate value, store in database & apc

The last step (generation) we're trying to handle as much as possible in cron jobs; but for some cases that hasn't been possible.

There's also a similar discussion at http://fourkitchens.com/blog/2009/11/30/intelligent-memcached-apc-intera...

My first concern with putting this in-cacherouter is the format of the configuration; in your example the only thing changing would be the engine type - i don't know if that would be enough for all cases? What about using a nested approach? I'd like to see cacherouter doing what David's suggesting on the four kitchens blog:

$conf['cacherouter'] = array(
  'default' => array(
    'engine' => 'apc',
    'tombstone' => array(
      'engine' => 'memcache',
      'servers' => array(),
      'shared' => TRUE,
    ),
  ),
);

This is very poorly implemented but initially good idea. In this patch, how do you distingush between "there is no cached value for this key" vs "this cache engine failed" ? You seem to be accessing both active cache engine and backup on FALSE - which is every "not found" key too. Note that you generally *don't want to* cache every value multiple times, especially when your backup engine is much slower cause added latencies may totally kill the point of having cache. Memcache API module DB-caching mode is a very poor idea imo, we should not repeat that error.
We need a backup in a case when a caching engine failed completely. I.e.: starting with empty backup cache - good (penalizing ourselves during crash recovery until backup cache is full again), pushing every cache entry multiple times - not good (penalizing ourselves all time).

Summary:
1) Implement a new way (new method ?) for cache engine to report it failed state.
2) Make some logging in CacheRouter to report that cache engine is broken. Logging into watchdog is ok, as long as you only log it *exactly once* when failed engine was initially discovered (to not flood logs with same messages).
Also I think there's no need to define any additional array keys (e.g. 'engine_backup') cause we already have "default" cache engine which should be enough as fallback. Overcomplicating this into making several backups is unneeded IMO.
UPDATE: On a second thought, engine_backup property (or similar) seems reasonable, for example in a case one has single "default" engine for all tables but still wants to have a backup for it.

We could also "mark" cache engine as failed and check it's status periodically. On success we could then return to it's usage and I think we could do it smoothly: for some time write to main engine, read both from main and backup engines. Then over time cache migrates to the main engine again and we stop using backup completely until next failure

There's some troubles with storage on early page cache with logging and storing variables because database unavailabality

Marked as duplicate #779602: check for apc/engine existence

It's a good proposal but it goes to hard when we need to check version of backend. For example bug caused by old version of APC which does not support apc_add() (some redhat products affected)

To summarize:
1) Need to implement a check backend availability, also check version of php module (some kind of hook_requirements)
2) Need to implement a configuration directive to setup backup engine
3) Put 1 and 2 into base class

so subscribing here. didn't recognize this title to be of similar topic. sure it is. :-)