Problem/Motivation
Note: this redundant-parsing pattern was identified with the assistance of Claude while investigating the root cause of the slow regeneration on our site; the finding was then manually reviewed and verified against real site content before being submitted here.
entity_usage ships five Track plugins that scan text/text_long/text_with_summary fields for embedded references: HtmlLink, CkeditorImage, EntityEmbed, MediaEmbed, and LinkIt (src/Plugin/EntityUsage/Track/). All five extend TextFieldEmbedBase and implement parseEntitiesFromText() by independently calling Html::load($text) to get a DOMDocument, then running their own DOMXPath query against it.
EntityUpdateManager::trackUpdateOnCreation()/trackUpdateOnEdition() loop over every enabled plugin for a given entity, and each plugin's trackOnEntityCreation() scans all of its referencing fields on that entity. In the default configuration (all plugins enabled), this means the same field value gets parsed into a DOM up to five separate times per entity revision (once per plugin), regardless of whether that plugin finds anything relevant in it.
This is wasteful on any site, but it compounds a known, nastier problem: Html::load()/DOMDocument::loadHTML() leaks memory in libxml's internal, process-global state that is never reclaimed at the PHP level (not even by gc_collect_cycles()). Only OS-level termination of the PHP process frees it. On sites with a large number of revisions (we're seeing ~650K node+paragraph revisions), running "Recreate entity usage statistics" (/admin/config/entity-usage/batch-update or drush entity-usage:recreate) processes every revision in one long-running process, so the 5x redundant parsing directly multiplies the rate of memory growth. In our case this made full regeneration slow and prone to exhausting memory, forcing workarounds (chunking the batch into smaller units with process restarts) just to complete a run.
Steps to reproduce
- On a default
entity_usageinstall, leave the defaulttrack_enabled_plugins(all plugins enabled), includinghtml_link,ckeditor_image,entity_embed,media_embed, andlinkit. - Create/save a node with a body field containing any HTML (a plain paragraph is enough; every enabled text-scanning plugin parses the whole field regardless of whether it matches anything).
- Add a temporary counter/breakpoint on
Html::load()(Drupal\Component\Utility\Html::load) and save the node, or rundrush entity-usage:recreate. - Observe
Html::load()is called once per enabled text-scanning plugin, per text field, per revision: 5 calls for the same field value instead of 1. - On a site with a large number of revisions, observe memory growth and total regeneration time scale accordingly.
Proposed resolution
Add a small, bounded cache to the shared TextFieldEmbedBase class so a given field's text is parsed into a DOMDocument once and reused by every plugin that needs it, instead of each plugin parsing it independently. Since every plugin only reads the DOM via its own DOMXPath query (none of them mutate it), sharing the parsed document across plugins is safe and produces identical tracking results. This is a pure performance change with no behavioral impact.
We've prototyped and locally tested a patch that:
- Adds a
protected static array $parsedDomCacheand agetParsedDom(string $text): \DOMDocumenthelper toTextFieldEmbedBase, keyed bymd5($text)and capped at a small number of entries (4) to keep memory bounded across a regeneration run touching hundreds of thousands of distinct field values. - Updates
HtmlLink,CkeditorImage,EntityEmbed,MediaEmbed, andLinkItto call$this->getParsedDom($text)inparseEntitiesFromText()instead ofHtml::load($text)directly.
We verified (via reflection against the live plugin instances): the cache correctly reuses the same DOMDocument object across different plugin instances for identical field text, correctly misses (and reparses) for different text, stays capped at 4 entries under sustained use, and running trackUpdateOnCreation() against real content behaves identically to before the change.
Remaining tasks
- Add automated test coverage: e.g. a kernel test asserting the number of
Html::load()/DOM-parse calls drops when multiple text-scanning plugins are enabled against the same field (via a spy/counter), and that tracked results are unchanged. - Get maintainer feedback on the caching approach and the cache-size bound (4 was chosen empirically to cover a field being parsed both with and without
HtmlLink'sfilter_urlpre-processing, see itsgetTextFromField()override, plus one or two additional distinct text fields per entity). - Confirm no regression in
trackOnEntityUpdate()/trackOnEntityDeletion()flows, and inEntityUsageTrackUrlUpdateInterfaceconsumers. - Decide whether the cache should live on the base class (as prototyped) or be pulled out into an injected, swappable service.
User interface changes
None.
API changes
Adds a new protected method TextFieldEmbedBase::getParsedDom() and a new protected static property TextFieldEmbedBase::$parsedDomCache. These are internal implementation details on an abstract base class, not part of the module's public API contract, so this shouldn't be a breaking change, but it's worth flagging since third-party code extending TextFieldEmbedBase directly (rather than through the plugin system) could theoretically already define a member with either name.
Data model changes
None. The entity_usage table schema is unchanged, and the tracked usage records produced by a regeneration are identical before and after this change (same entities/references detected, same counts).
Issue fork entity_usage-3615226
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #5
luongosb commentedComment #6
luongosb commentedApologies for all of the updates on this issue. I'm still getting the hang of using Drupal's issue queue :)
I wasn't sure if I should create two seperate issues for the 5.x and the 2.x branch, so I attached both MRs to this issue.
Please let me know if you have any feedback! Thanks!
Comment #7
luongosb commentedComment #8
luongosb commented