Problem/Motivation

Since we have cache tags, there is aconstant debate about how fine-grained they should be. Both has up- and down-sides.
Fine-grained cache tags allow fine-grained invalidation, but every cache tags adds a probabilistic fraction of one query as cost in the CacheTagsChecksumInvalidator service.

Proposed resolution

Add a CacheTagsAggregator service that aggregates a set of fine grained cache tags to lower granularity.
So get the best of both worlds: Have fine-grained cache tags in the internal API, while only a subset of those cache tags acutally hit the database. Plugability allows site specific optimization of cache tag aggregation strategy.

Remaining tasks

Implement, bikeshed, commit.

User interface changes

None.

API changes

Pure API addition.

Data model changes

None.

Issue fork drupal-3358701

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

geek-merlin created an issue. See original summary.

geek-merlin’s picture

Title: (CORE) Add a CacheTagsAggregator » Add a CacheTagsAggregator
Project: Geek Merlin's Issue Incubator » Drupal core
Version: » 10.1.x-dev
Component: Code » cache system

geek-merlin’s picture

Status: Active » Needs review

Some POC code to discuss the direction this goes.

andypost’s picture

Status: Needs review » Needs work

needs to fix CS

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

berdir’s picture

I don't really understand the goal of this. Some real use cases of when you'd want to use this could help,

The example is technically valid, but it's not something you ever want to do. Ending up with too many node:list cache tags on your site is one of the main performance issues.

Invalidation and checking also doesn't follow the same rules. Going with the example, When invalidating, we typically invalidate node:ID, node:list, node:list_article and possibly more. And we must invalidate all of them. When checking for invalidation, I could imagine a case where we say if an item has node:list and node:ID then we can ignore node:ID, but with the bundle cache tags, that gets complicated/expensive again.

Maybe a scenario where you have 10 global config cache tags that all add some stuff to global attachments (like google analytics and others), those you could aggregate to a single one, but that will be very site specific and you'd need to hardcode that, because any sort of configuration for that will most likely be more expensive to check than an extra query or even just a more entries in an IN condition.

catch’s picture

I think this could be useful for headers, but we'd probably want to use a de-duplication strategy similar to how cache contexts work, not always replace.

There are various issues around like #2844620: Automatically split cache debug headers into multiple lines when they exceed 8k and #2952277: Minify the cache tags sent in the header and related issues trying to deal with this.

We have similar logic in CacheContexts::optimizeTokens() - when a cache object has both user.permissions and user cache contexts, it will change this to only user because user.permissions is implied.

With cache tags, if we have node:1, node:123 and node:list, we can optimize that down to just node:list in the headers. When entities are saved, the list cache tags are invalidated, so we know the CDN will get the invalidation every time.

Then when the response doesn't have node:list there are two choices - optimize it down anyway to reduce the header size (and potentially serve the HTML faster), or skip the optimization and have overly aggressive cache invalidation at the CDN level.

Since core doesn't serve these headers by default, and because the debug cache headers need to be correct, this wouldn't really be usable in core itself, but given the number of modules that are trying to do similar things, it should probably either be in core or a shared dependency module they all use.

geek-merlin’s picture

@catch #9: Interesting, but this is NOT the main use case.

Let me make a use case on a custom site:
You have a site with 80 node types, and core maintains 1 list cache tag for each and every one of them. Maybe you don't care at all. Or maybe you want one cache tag for 3 of them and a second cache tag for the other 77. The POC aggregator lets you do that with minimal effort.

The big picture: We have those constant wars on more cache tags here, less cache tags there.
Some developer use cases profit from more fine-grained cache tags (in the API), and in the case of bundle list cache tags, core said yes.
But for performance, more cache tags (in the DB) are bad.

The approach allows to have the cake and eat it. And to stop that war:
Add more cache tags in the API and make developers happy. And to map them to a smaller set in the DB to make performance owners happy.

Maybe this makes it clear.

catch’s picture

@geek-merlin, since Drupal 11.3 the number of cache tag lookups on most pages has approximately halved or less. e.g. #3505248: Ability to preload frequently used cache tags, #1237636: Lazy load multiple entities at a time using fibers and several other issues. More details on https://www.youtube.com/watch?v=qgmGtKJClRM and in https://www.md-systems.ch/en/blog/2025-12-16/performance-improvements-dr.... So a lot of the previous trade-offs between fine grained cache tags vs. lookups are becoming moot as regards cache backend performance.

geek-merlin’s picture

Yup, i noticed that, and it is very good. I've just seen that an awful lot of complexity has been added to the system to eradicate single cache tags, and wanted to provide an alternative, to help keep code maintainable, have no more complexity introduced, and maybe roll back some complexity and use cache tags again. Apart from that, i have no interest in pushing this solution further.
So if someone wants to pick it up and do some benchmarks, fine, if this rots forever as POC, fine for me too.

geek-merlin’s picture

Title: Add a CacheTagsAggregator » Add a CacheTagsAggregator to reduce DB cache tags while keeping API cache tags fine-grained
berdir’s picture

You haven't replied to #8, I still don't really understand how you imagine specific use cases and how this would help. I tried to explain why just aggregating everything node:* to node_list is very much not what you want, as saving any node would then always invalidate every single rendered node (I get it's just an example, but it's not a good one). Maybe you could get rid of a handful of more specific cache tags, but especially with redis 2.x, which consistently uses a pipeline now for multiple invalidations, there's minimal difference between invalidating 1 and 5 cache tags at once now. The database backend still uses one query per tag, but you probably shouldn't use database if you care about performance and even then, invalidations aren't really the main concern, there's already static caching to not invalidate tags more than once per request.

The global cache tag use case I mentioned there is solved by https://www.drupal.org/node/3505248, tags that are used on most/many pages can be put into that list and then they're prefetched on every page in a single SELECT/mget. It's not automatic apart from the one that core ships with by default, but if you care, it's easy to opt into that, at least as easy as writing such an aggregator.

Another example is block cache tags, those are very repetitive and long, but I think it's better to get rid of them at the source which I'm trying in #3341042: Use only the list cache tag for block config entities but there are a few edge cases that something like this would run into as well.

Correctly used, specific cache tags are a good thing and valuable, but it's something that needs to be thought about and a tradeoff to be found. As mentioned, invalidations are relatively cheap, and either a cache tag is actively used, then it generally shouldn't be aggregated, or not, and then it's existence in the database is an absolutely tiny cost.

Re #9, what I once did a very long time ago in our now defunct news distribution is a custom implementation that used a combination of hardcoded rules and very short hashes to compress cache tags, so node:1 became n:1, and config:block.block.some_long_block_id became something like c:b:abc. That cut down significantly on the header size and mostly solved the issue for us. That was before purge and related projects existed.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.