Perhaps it would be nice to handle different cache bins in multiple Redis DBs.

Why? Currently Redis doesn't support wildcard delete so the Redis module have to clear all keys one by one for a cache clear of one cache bin. This could be possible failed with an PHP execution timeout, when you Redis Cache have many many keys. With a seperate Redis DB for one cache bin the module could use a flushdb to clear a hole cache bin.

Problem: How to configure this and are their any performance issues, when you have every time switch between the different Redis DBs?

Perhaps someone have any ideas on that. We have tried a patch with a LUA script handling for a wildcard delete, but then our Redis Server sometimes could fail after a flush all caches because it is also busy from the LUA script. Patch can be found here #2140897-43: cache_clear_all() not properly handled in PhpRedis.php

CommentFileSizeAuthor
#5 redis-Cache_bin_handling_in_multiple_Redis_DBs-2395255-5.patch42.43 KBsomersoft
#4 redis-Cache_bin_handling_in_multiple_Redis_DBs-2395255-4.patch41.19 KBsomersoft
#2 redis-cache_bin_redis_db_handling_predis-2395255-2.patch2.7 KBIT-Cru
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

pounard’s picture

pounard
Primary language French
CreditAttribution: pounard commented

This is an idea, maybe I could try. But I think there is something inconsistent in using multiple databases, sound not really consistent IMHO, but why not, I could still leave the possibility to configure this.

IT-Cru’s picture

IT-Cru
Primary language English
Location Munich
CreditAttribution: IT-Cru commented
FileSize
redis-cache_bin_redis_db_handling_predis-2395255-2.patch2.7 KB

I've created a patch for Caching with predis lib. Perhaps not the best way to place the Redis DB switching code for cache bins, but a possible starting point.

settings.php example for mapping cache bins to Redis DBs (Redis only supports numeric "names" for its DBs). All other cache bins will be placed in Redis DB 0 or configured 'redis_client_base'.

$conf['redis_client_base_bins']['cache_bootstrap'] = 1;
$conf['redis_client_base_bins']['cache_bip'] = 2;
$conf['redis_client_base_bins']['cache_entity_file'] = 3;
$conf['redis_client_base_bins']['cache_entity_node'] = 4;
$conf['redis_client_base_bins']['cache_entity_taxonomy_term'] = 5;
$conf['redis_client_base_bins']['cache_field'] = 6;
$conf['redis_client_base_bins']['cache_views'] = 7;
pounard’s picture

pounard
Primary language French
CreditAttribution: pounard commented
Status: Active » Closed (won't fix)

For your information, I won't fix this because the module is going another direction, I'm trying to make it proxy-shardable (which is working very nicely with the 7.x-3.x branch) - Using multiple databases is not really a planned feature.

If you still have timeout problems, you have two solutions:

  • If you use only a single one Redis instance, you can safely upgrade the Redis module 2.x version to the latest stable, and add $conf['redis_eval_enabled'] = true; in your settings.php file, and it'll work gracefully.
  • If you use more than one Redis instance throught a sharding proxy, or if you have such high data volume that even the EVAL powered 2.x version is slow, consider switching the to the 3.x branch that works differently, add $code['redis_flush_mode'] = 3; to your settings.php file, and should work gracefully.
somersoft’s picture

somersoft CreditAttribution: somersoft commented
Version: 7.x-2.12 » 8.x-1.x-dev
Status: Closed (won't fix) » Needs review
FileSize
redis-Cache_bin_handling_in_multiple_Redis_DBs-2395255-4.patch41.19 KB

This patch enables the ability to use a multiple Redis services and configure which cache bin use which service based upon the cache bin name.
The Redis report has been greatly modified to report information per service. It also lists which cache bins have been configured to use each service.
The patch has been used with AWS services in production.
It is fully backward compatible. for single service use and there are no changes to limit the configuration options.
Documentation has also been updated to indicate how to add and configure multiple services.

I have also used these patches
Currently Drush Cr or Cache Clear UI does not flush Redis cache - 2765895
Source: https://www.drupal.org/files/issues/redis-n2765895-16.patch

Support TLS for Predis - 3004561
Source: https://www.drupal.org/files/issues/2021-11-19/redis-support_tls_on_pred...

Implement initial RedisCluster client integration - 2900947
Source: https://www.drupal.org/files/issues/2022-11-10/2900947-57.patch

Error: Call to a member function hMset() on bool - 3143396
Source: https://www.drupal.org/files/issues/2022-01-18/3143396-4-fix-pipe.patch

and 2900947-57.patch does not apply cleanly. This patch assumes that 2900947-57.patch has been applied.

somersoft’s picture

somersoft CreditAttribution: somersoft commented
FileSize
redis-Cache_bin_handling_in_multiple_Redis_DBs-2395255-5.patch42.43 KB

Updated the patch for redis 1.7
Here is the list of patches from composer.patches.json

    "drupal/redis": {
      "Currently Drush Cr or Cache Clear UI does not flush Redis cache - https://www.drupal.org/project/redis/issues/2765895": "https://www.drupal.org/files/issues/redis-n2765895-16.patch",
      "Support TLS for Predis - https://www.drupal.org/project/redis/issues/3004561": "https://www.drupal.org/files/issues/2021-11-19/redis-support_tls_on_predis-3004561-37.patch",
      "Implement initial RedisCluster client integration - https://www.drupal.org/project/redis/issues/2900947": "https://www.drupal.org/files/issues/2024-01-15/2900947-60.patch",
      "Cache bin handling in multiple Redis DBs - https://www.drupal.org/project/redis/issues/2395255": "https://www.drupal.org/files/issues/2024-02-01/redis-Cache_bin_handling_in_multiple_Redis_DBs-2395255-5.patch"
    },