Problem/Motivation

If a large amount of cache tags are stored in \Drupal\Core\Cache\CacheTagsChecksumTrait::$tagCache, then the the array operations we run on them, such as array_keys() and array_diff() can get very expensive.

I doubt that this makes a measurable impact on regular operations as there aren't *that* many cache tags, but in special cases, such as the new redis report, it adds up *a lot*. The report loops over all caches and checks if they haven't been invalidated, to figure out a percentage of expired and invalidated items.

Turns out, the results in massive costs in side calculateChecksum(), which I managed to catch with the blackfire continuous monitoring report:

It's somewhat similar to the memory cache cache tag invalidation issues.

Note: the original performance measurements were taken on 11.3.x and mostly already fixed with #3580109: Optimise CacheTagsCheckSumTrait::calculateChecksum(), which I had forgotten about.

Steps to reproduce

Proposed resolution

Convert the remaining array_diff/array_merge/array_keys()/.. calls to loops over possibly much smaller $tags and $this->preloadTags.

Performance improvement is small but I think still worth it to avoid the extra array operations on filling the static cache with 0's, also avoids an extra combined variable.

Remaining tasks

User interface changes

Introduced terminology

API changes

Data model changes

Release notes snippet

CommentFileSizeAuthor
Screenshot From 2026-08-06 11-43-53.png47.09 KBberdir

Issue fork drupal-3615175

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

berdir created an issue. See original summary.

berdir’s picture

berdir
Primary language German
Location Switzerland
commented
Title: Array functions in CacheTagsChecksumTrait::calculateChecksum() » Slow array functions in CacheTagsChecksumTrait::calculateChecksum()

sjpagan made their first commit to this issue’s fork.

sjpagan opened merge request !16604

sjpagan’s picture

sjpagan commented
Status: Active » Needs review

MR !16604, pipeline green.

The cost is in registerCacheTagsForPreload(). array_merge() copies the whole registered list on every call, and DatabaseBackend::getMultiple() calls it once per lookup.

Bench: N batches of 10 tags, then one checksum read.

1,000    22 ms       -> 3 ms
10,000   3,669 ms    -> 35 ms
50,000   143,108 ms  -> 218 ms

Linear after the change, quadratic before.

The calculateChecksum() conversions alone changed nothing measurable. array_combine() with += was slower than the current code: += on a property duplicates the array.

$preloadTags is now keyed by tag. Nothing in core or contrib reads it, including RedisCacheTagsChecksum.

smustgrave’s picture

smustgrave commented
Status: Needs review » Needs work
Issue tags: +Needs tests

@sjpagan thanks for contributing but just a few notes.

As previously commented some of the comments have been coming off as AI so please take a moment and read https://www.drupal.org/docs/develop/issues/issue-procedures-and-etiquett...

There's no rule against using AI at all but again please read the issue etiquette page.

Thanks.

sjpagan’s picture

sjpagan commented

Okay @smustgrave, I’ve read the article, i wasn’t aware of that, so I’ll be more careful when writing comments and code, following the guidelines.
I saw that you replied to my other questions as well; do I need to reply to those too regarding the same issue?

smustgrave’s picture

smustgrave commented

No sir you are good :) thank you

sjpagan’s picture

sjpagan commented
Status: Needs work » Needs review

Added the missing test coverage. `CacheTagsChecksumTest` and `CacheTagsChecksumHelper` cover the preload path, which had no test in core. They pass both before and after the change.

Trimmed the comments the previous commits added to the trait, from 17 lines to 7. No code changes in that commit. Also updated the MR description.

smustgrave’s picture

smustgrave commented

Going to let other reviews these as the bot seems to be putting a lot that don't have time to review all but first thing is to check if existing test coverage exists already vs new files. Being able to add a new assert to an existing test that's related is preferred.

If test coverage is missing it may not be in scope to add here but again will leave for others.

sjpagan’s picture

sjpagan commented

Checked first — nothing in core calls `registerCacheTagsForPreload()`, and there's no test class for the trait. The only separate helper file in core is `CacheCollectorHelper`, and that one is shared by three test classes, so it didn't apply here.

Merged it into `CacheTagsChecksumTest.php` as a second class instead. One new file now, not two.

berdir’s picture

berdir
Primary language German
Location Switzerland
commented
Status: Needs review » Needs work
Issue tags: -Needs tests

Confused on the timeline of the comments here, feels like some are replies to others but were written first, but maybe I just didn't read them properly.

I set this to a bug report, but in the sense of "a performance bug", not a functional one. I guess we usually treat those as tasks. This contains no behavior change, it's purely an internal optimization. As such, I don't think this needs any tests at all. Our existing tests *should* be sufficient to cover the changed code, as commented on the MR.

berdir’s picture

berdir
Primary language German
Location Switzerland
commented
Status: Needs work » Reviewed & tested by the community

Thanks, I think this looks good now. Per the numbers in #5, not sure how common it is to have more than 1k cache tag lookups in the cache, but there's no downside to this I think and for long running processes like migrations, this could easily add up quite a bit.

sjpagan’s picture

sjpagan commented

Withdrawing the numbers in #5: that bench never reads a checksum between
registrations. registerCacheTagsForPreload() is back to main.

Locally the loops in calculateChecksum() give 8 to 14 percent, flat from
1,000 to 50,000 tagCache entries.

berdir’s picture

berdir
Primary language German
Location Switzerland
commented
Category: Bug report » Task
Priority: Normal » Minor
Issue summary: View changes

I'm sorry, I made a mistake and didn't review and test this well enough.

The performance data I reported was based on 11.3.x, not main. This problem I saw there was already fixed in #3580109: Optimise CacheTagsCheckSumTrait::calculateChecksum(), I was part of that issue, I just forgot about it.

I only realized this when double-checking the performance tests and then was confused when I didn't see much of a difference, even with very large tag lists, not as I expected it at least.

I considered closing this issue as a duplicate, but I think the remaining changes are useful, more readable/simpler and a tiny bit faster, but only when preload tags are involved and actual lookups, specifically lookup misses.

catch’s picture

catch
he/him
Primary language English
commented
Title: Slow array functions in CacheTagsChecksumTrait::calculateChecksum() » Tidy up CacheTagsChecksumTrait::calculateChecksum()
catch’s picture

catch
he/him
Primary language English
commented
Status: Reviewed & tested by the community » Fixed

OK agreed, even if the performance gains are very marginal, the code is a bit easier to follow, so no reason not to do this.

Committed/pushed to main. Doesn't apply to 11.x but given there's not a noticeable performance improvement I think we could leave this in main. Thanks!

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.

  • catch committed 946e35e4 on main 
    task: #3615175 Tidy up CacheTagsChecksumTrait::calculateChecksum()

By:...