Problem/Motivation
The audit_cache submodule reads the entire {cachetags} table into memory
without any limit, in CacheAnalyzer::readCacheTagsFromDatabase():
$result = $this->database->query(
'SELECT [tag], [invalidations] FROM {cachetags}'
);
...
foreach ($result->fetchAll() as $row) {
$tags[$row->tag] = (int) $row->invalidations;
}
The {cachetags} table (Drupal core's DatabaseCacheTagsChecksum) can grow
without an upper bound. CacheAnalyzer itself already acknowledges this
risk, defining bloat thresholds:
protected const TAG_COUNT_WARNING_THRESHOLD = 5000;
protected const TAG_COUNT_ERROR_THRESHOLD = 50000;
However, the actual data read is not capped before those thresholds are
even evaluated. On sites where {cachetags} has grown far beyond
TAG_COUNT_ERROR_THRESHOLD (e.g. several hundred thousand or millions of
rows — which can easily happen on long-running sites with volatile
custom entities), simply attempting to load the full table into a PHP
array exhausts the configured memory_limit before the analyzer can even
report the bloat:
Fatal error: Allowed memory size of 1073741824 bytes exhausted
(tried to allocate 83886080 bytes) in
.../audit_cache/src/Plugin/AuditAnalyzer/CacheAnalyzer.php on line 1813
Because this is an uncaught PHP fatal error (not a handled exception),
it can interrupt the whole audit run depending on how it's invoked (UI,
Drush, cron), not just the cache tags section. This is particularly
relevant since the module is positioned for pre-deploy validation and
recurring audits, where a fatal error blocks the entire workflow.
Steps to reproduce
1. Install/enable the "audit" module with the "audit_cache" submodule on
a site using the core database backend for cache tag checksums
(Drupal\Core\Cache\DatabaseCacheTagsChecksum).
2. Grow the {cachetags} table well beyond ~100,000 rows (e.g. via a bulk
SQL insert of synthetic tags, or on a long-running site with volatile
custom entities that naturally accumulates rows over time).
3. Run the audit_cache analysis (via the module's UI or Drush).
4. Observe a PHP fatal error:
"Allowed memory size of X bytes exhausted (tried to allocate Y bytes)
in .../audit_cache/src/Plugin/AuditAnalyzer/CacheAnalyzer.php"
instead of a completed report (or a controlled warning about table
bloat).
Proposed resolution
Cap the amount of data enumerated from {cachetags} while keeping the
bloat report accurate:
1. Add a hard safety cap constant, separate from the existing
warning/error thresholds:
protected const TAG_COUNT_HARD_CAP = 100000;
2. Rewrite readCacheTagsFromDatabase() to use the DBTNG query builder
with range() instead of a raw, unlimited SELECT + fetchAll():
$result = $this->database->select('cachetags', 'ct')
->fields('ct', ['tag', 'invalidations'])
->range(0, self::TAG_COUNT_HARD_CAP)
->execute();
This applies a real LIMIT at the database level (not just a PHP-side
break), and iterates directly over the Statement instead of
duplicating the whole result set via fetchAll().
3. Add a new lightweight method to get the true row count without
loading any rows into memory:
protected function getCachetagsRowCount(): int {
return (int) $this->database->query(
'SELECT COUNT(*) FROM {cachetags}'
)->fetchField();
}
4. In analyzeCacheTagInvalidations(), use the real row count (falling
back to the enumerated count when larger) so the "table bloat"
warning/error stays accurate even though enumeration is capped:
$db_row_count = $this->getCachetagsRowCount();
$bloat = $this->analyzeTagTableBloat(
max($current_raw_count, $db_row_count)
);
This is backward compatible: no public method signatures or result
formats change, aside from the new internal hard cap on how many
individual tag rows are enumerated for the detailed listing. Redis and
Memcache cache tag backends are unaffected, since they use separate
code paths (readCacheTagsFromRedis(), etc.).
This fix has been implemented and tested locally, and will be submitted
as a merge request against the project's default branch (via an issue
fork), following the current Drupal.org/GitLab contribution workflow.
A patch can also be provided on request if preferred for review.
Remaining tasks
- Review and confirm the approach with the maintainer.
- Decide on the exact value for TAG_COUNT_HARD_CAP (100000 proposed;
open to adjustment based on maintainer's guidance or typical site
sizes).
- Add/port automated test coverage if the module has an existing test
suite for CacheAnalyzer.
- Update in-code documentation/comments if needed per project
conventions.
User interface changes
None. The structure of the reported results (result items, messages)
is unchanged. On sites where {cachetags} exceeds the new hard cap, the
"table bloat" message will report the accurate total row count (via
COUNT(*)) even though the detailed per-tag listing is capped.
API changes
None public. Internal method readCacheTagsFromDatabase() changes its
implementation (raw SQL query -> query builder with range()), and a new
protected method getCachetagsRowCount() is added. No public method
signatures change.
Data model changes
None.
| Comment | File | Size | Author |
|---|---|---|---|
| #4 | 3621704-audit_cache-cachetags-oom-fix.patch | 3.13 KB | jisladev |
Issue fork audit-3621704
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #2
jisladev commentedComment #4
jisladev commentedAttached is a patch that has been applied and tested on Drupal 10.6.16
Comment #5
jisladev commentedComment #6
jisladev commentedComment #8
trebormcThanks for the report and the MR. I could reproduce it: with one million rows in
cachetags, the Cache audit ran out of memory.The patch fixes the memory error, but the capped read takes any 100,000 rows. On a bloated table almost all of them are entity-specific tags (
node:42,user:1...), which the analysis discards right after reading them, so the tags that are actually analyzed were dropped at random. In my test only 1 of 148 relevant tags was left.So I kept the
COUNT(*)for the bloat check and the cap, and changed the query to filter out entity-specific tags withNOT REGEXP(supported by every core database driver) and to read the most invalidated tags first. With one million rows the audit now finishes in under a second, reports the real row count and analyzes all the relevant tags. I also added a kernel test.While looking into this I found two related problems with other backends. With Redis or Memcache, the
cachetagstable only holds leftover rows from an earlier backend, but it was still analyzed as current data; now it's only analyzed with the database backend, and otherwise its leftover rows are just reported as information. And with Redis, all the cache tag keys were read at once with a singleMGET, which could run out of memory in the same way; they are now read in batches, skipping entity-specific tags and with the same cap.I'm marking this as Fixed: it's merged into the development branch and will be included in the next stable release.