Problem/Motivation

The Redis Report available under /admin/reports/redis is a great tool. But wouldn't it be nice to be able to access those data via the command line, with Drush?

Steps to reproduce

Proposed resolution

Add support for accessing the data from the Report page via Drush.

Remaining tasks

Decide if it would be an improvement.
Evaluate if adding Drush support is a big task, or not.

User interface changes

API changes

Data model changes

Issue fork redis-3555973

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

ressa created an issue. See original summary.

berdir made their first commit to this issue’s fork.

berdir opened merge request !82

berdir’s picture

berdir
Primary language German
Location Switzerland
commented
Status: Active » Needs work

Yes, I've been thinking about this for a while and finally got something started. Unfinished, but this already does have some very useful bits that the web report doesn't have, because I'm more comfortable doing something slower here as I have a progress bar and no/higher execution limits.

Specifically, it extend the bin info to include the size (as reported by redis with MEMORY USAGE) as well as the number of expired and invalidated cache entries:

ddev drush redis:report
Client            : PhpRedis
Version           : Valkey 8.1.3
Mode              :
Connected_clients : 2
Keys              : 3552
Memory            : 13.02M / 2 GB (0%), maxmemory policy: allkeys-lfu
Uptime            : 3 hours 11 min
Read_write        : 353.65 MB read (84%), 66.65 MB written (16%), 307267 commands in 810 connections.

 ┌ Calculating cache statistics ────────────────────────────────┐
 │ ████████████████████████████████████████████████████████████ │
 └─────────────────────────────────────────────── 3,500 / 3,552 ┘

Bin statistics:
 -------------------- ------- ----------- --------- --------------
  Bin                  Count   Size        Expired   Invalidated
 -------------------- ------- ----------- --------- --------------
  page                 159     3.16 MB     0         147 (92.45%)
  render               1432    2.73 MB     0         164 (11.45%)
  dynamic_page_cache   218     2.43 MB     0         120 (55.05%)
  data                 474     597.66 KB   0         151 (31.86%)
  discovery            169     508.21 KB   0         0
  default              258     496.65 KB   0         0
  config               568     398.95 KB   0         0
  entity               167     318.27 KB   0         2 (1.2%)
  bootstrap            12      91.98 KB    0         0
  menu                 81      82.36 KB    0         0
  static               0       0 bytes     0         0
  access_policy        0       0 bytes     0         0
  toolbar              0       0 bytes     0         0
 -------------------- ------- ----------- --------- --------------


Most frequently invalidated cache tags:
Node_list:article                     : 1
Content_moderation_state:18:revisions : 1
Node_list                             : 1
4xx-response                          : 1
Search_index:node_search              : 1
Node:18:revisions                     : 1
Content_moderation_state:18           : 1
Node:18                               : 1
Content_moderation_state_list         : 1
Search_index:help_search              : 1

"Fun" fact: This is the result of umami on Drupal core 11.x, a "wget --mirror" to populate the cache and then saving a single article node. This directly invalidates 90% of the page cache, 50% of the dynamic page cache and 30% of the data cache (views results). That's due to the invalidation of the node_list cache tag. Shows nicely that if you have semi-frequent content updates and use views on many pages in blocks and so on, it's crucial to avoid the node_list cache tag and use something like views_custom_cache_tag.

berdir’s picture

berdir
Primary language German
Location Switzerland
commented
Title: Support Report via Drush? » Support Report via Drush
berdir’s picture

berdir
Primary language German
Location Switzerland
commented
Status: Needs work » Needs review

Pretty much completed what I wanted to do with this, cache tags now also include number of usages in cache entries and added render cache varations and warnings:

Client            : PhpRedis
Version           : Valkey 8.1.3
Connected_clients : 2
Keys              : 3628
Memory            : 13.13M / 2 GB (0%), maxmemory policy: allkeys-lfu
Uptime            : 2 hours 26 min
Read_write        : 329.3 MB read (85%), 56.04 MB written (15%), 265949 commands in 118 connections.

 [warning] Using an allkeys eviction policy may drop cache tag checksums or other persistent data, consider using a volatile eviction policy, see the module documentation (https://project.pages.drupalcode.org/redis/) and eviction documentation (https://valkey.io/topics/lru-cache#eviction-policies).

 ┌ Calculating cache statistics ────────────────────────────────┐
 │ ████████████████████████████████████████████████████████████ │
 └─────────────────────────────────────────────── 3,600 / 3,628 ┘

Bin statistics:
 -------------------- ------- ----------- --------- --------------
  Bin                  Count   Size        Expired   Invalidated
 -------------------- ------- ----------- --------- --------------
  page                 159     3.16 MB     0         147 (92.45%)
  render               1469    2.81 MB     0         172 (11.71%)
  dynamic_page_cache   220     2.45 MB     0         121 (55%)
  data                 495     638.38 KB   0         152 (30.71%)
  discovery            169     507.65 KB   0         0
  default              267     489.63 KB   0         0
  config               570     397.66 KB   0         0
  entity               168     319.42 KB   0         4 (2.38%)
  menu                 81      82.3 KB     0         0
  bootstrap            11      70.59 KB    0         0
 -------------------- ------- ----------- --------- --------------


Render cache items with many variations
 --------------------------------------------------------------- ------- ------------------------------------------------------------------------------------------
  Key                                                             Count   Contexts
 --------------------------------------------------------------- ------- ------------------------------------------------------------------------------------------
  entity_view:block:umami_local_tasks                             232     route, languages:language_content, layout_builder_is_active:node,
                                                                          layout_builder_is_active:taxonomy_term, languages:language_url, user.roles:authenticated
  entity_view:block:umami_breadcrumbs                             191     languages:language_url, route, url, languages:language_content
  entity_view:block:umami_help                                    120     route
  entity_view:block:umami_views_block__articles_aside_block_1     64      languages:language_content, url, user.node_grants:view, languages:language_url,
                                                                          route.name.is_layout_builder_ui, timezone
  entity_view:block:umami_main_menu                               12      languages:language_url, route.menu_active_trails:main
  toolbar                                                         8       languages:language_url, route, user.roles:anonymous
  entity_view:block:claro_secondary_local_tasks                   8       route, languages:language_content, layout_builder_is_active:node, languages:language_url
  entity_view:block:claro_primary_local_tasks                     8       route, languages:language_url, languages:language_content, layout_builder_is_active:node
  views:fields:recipe_collections:block:fdb27fba4d5f968ea52227f   6       languages:language_url
  521bd1414d24eab3c70a5cf7043a52b3309c26fc8
  views:fields:recipe_collections:block:a1594c7341ea125112a00cb   6       languages:language_url
  a4e101ac55b58e600e49b4299d0a065262a515202
  views:fields:recipe_collections:block:2e2fd2a8db60a884b1c7457   6       languages:language_url
  9779035a065d46ac95e05eccc2c272bfcd0ee1835
  entity_view:block:umami_footer_promo                            6       languages:language_content, languages:language_url, route.name.is_layout_builder_ui
  views:fields:recipe_collections:block:ebf6a3f162c339d7024f769   6       languages:language_url
  d1ddc7eeaa152e00027f1216bb9b697e929c845f0
  entity_view:block:umami_branding                                6       languages:language_url
  views:fields:recipe_collections:block:54a8283ca1b5db44dbfb82e   6       languages:language_url
  c37348be0e55168b777b636f87db2a619c9a91040
  view:frontpage:display:page_1                                   6       languages:language_url, route.name.is_layout_builder_ui, timezone, user.node_grants:view
  views:fields:recipe_collections:block:c96be8074bfbccf8cc28c87   6       languages:language_url
  59bdd19b3cc9c82438ebaaeb686f0d5cf6646ca13
  entity_view:block:umami_banner_home                             6       languages:language_content, languages:language_url, route.name.is_layout_builder_ui
  entity_view:block:claro_local_actions                           6       route, languages:language_url
  views:fields:recipe_collections:block:c324ffe415b4611f3ba65b9   6       languages:language_url
  a6b8d863a39bf0a0a98e191d80f3da1ef1ae58e0b
 --------------------------------------------------------------- ------- ------------------------------------------------------------------------------------------


Most frequently invalidated cache tags:
 --------------------------------------- --------------- --------
  Tag                                     Invalidations   Usages
 --------------------------------------- --------------- --------
  node_list                               2               569
  content_moderation_state_list           2               0
  4xx-response                            2               0
  search_index:node_search                2               4
  node:18                                 1               167
  node_list:recipe                        1               0
  content_moderation_state:3:revisions    1               1
  node:3                                  1               128
  content_moderation_state:18:revisions   1               1
  node:18:revisions                       1               1
  node:3:revisions                        1               1
  content_moderation_state:3              1               0
  content_moderation_state:18             1               0
  node_list:article                       1               0
  search_index:help_search                1               0
 --------------------------------------- --------------- --------
berdir’s picture

berdir
Primary language German
Location Switzerland
commented
Status: Needs review » Needs work

Did some testing with a backported version of this on a production project. it can take a while to collect the data, 120s for us with 100k keys.

However, it exposed a bug with the scan implementation, it may return an empty list, but the loop must be continued until the iterator is 0 again. see https://github.com/phpredis/phpredis?tab=readme-ov-file#scan. there's also the automatic retry option,

ressa’s picture

ressa
he/him
commented

The examples in your last comments look great @berdir, thanks! And also nice that additional bits of info might be included, like the size, expired and invalidated cache entries as well.

Interesting observation and insight about cache invalidation, thanks for sharing. Just for those who are not sure (like myself) but how would you easiest avoid using node_list cache tag, and use something like views_custom_cache_tag instead -- can it be set in the View, or some other way?

It was on my list to try out the MR for the new Drush command, but I'll wait until the Status changes.

berdir’s picture

berdir
Primary language German
Location Switzerland
commented

https://www.drupal.org/project/views_custom_cache_tag should explain fairly well on how to use that, #support might also be to able with specific questions. Yes, essentially you configure it in the view and instead of invalidating all node views every time any node is saved, you just invalidate article lists if it's an article.

ressa’s picture

ressa
he/him
commented

Very cool module, thanks for making it and a fast reply, I really appreciate it. It might be a Drupal core candidate, since invalidating cache tags for all nodes seem to broad ... but that can always happen in the future eventually, if the number of installs continue to grow.

berdir’s picture

berdir
Primary language German
Location Switzerland
commented
Status: Needs work » Needs review

We're getting off topic, but while I think views_custom_cache_tag should be used on all non-trivial sites with lots of content and updates, it's a developer tool. It's not rocket science, but you need to understand cache tags and add the correct strings by hand. There's #3055371: Use new cache tag ENTITY_TYPE_list:BUNDLE in Views to improve cache hit rate to do that automatically in core for node types, but there are many possible configurations in views, getting this right is hard, and if it's wrong then that means that lists aren't updating when they should, which can be a major problem.

The scan issue should be fixed.

  • berdir committed 74cb6ba3 on 2.x 
    feat: #3555973 Support Report via Drush

By: berdir
berdir’s picture

berdir
Primary language German
Location Switzerland
commented
Status: Needs review » Fixed

Merged, added some docs and hints on how to use this information.

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.

ressa’s picture

ressa
he/him
commented

Fantastic, thank you! I am looking forward to have this tool available.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.