Problem/Motivation

Note: this redundant-parsing pattern was identified with the assistance of Claude while investigating the root cause of the slow regeneration on our site; the finding was then manually reviewed and verified against real site content before being submitted here.

entity_usage ships five Track plugins that scan text/text_long/text_with_summary fields for embedded references: HtmlLink, CkeditorImage, EntityEmbed, MediaEmbed, and LinkIt (src/Plugin/EntityUsage/Track/). All five extend TextFieldEmbedBase and implement parseEntitiesFromText() by independently calling Html::load($text) to get a DOMDocument, then running their own DOMXPath query against it.

EntityUpdateManager::trackUpdateOnCreation()/trackUpdateOnEdition() loop over every enabled plugin for a given entity, and each plugin's trackOnEntityCreation() scans all of its referencing fields on that entity. In the default configuration (all plugins enabled), this means the same field value gets parsed into a DOM up to five separate times per entity revision (once per plugin), regardless of whether that plugin finds anything relevant in it.

This is wasteful on any site, but it compounds a known, nastier problem: Html::load()/DOMDocument::loadHTML() leaks memory in libxml's internal, process-global state that is never reclaimed at the PHP level (not even by gc_collect_cycles()). Only OS-level termination of the PHP process frees it. On sites with a large number of revisions (we're seeing ~650K node+paragraph revisions), running "Recreate entity usage statistics" (/admin/config/entity-usage/batch-update or drush entity-usage:recreate) processes every revision in one long-running process, so the 5x redundant parsing directly multiplies the rate of memory growth. In our case this made full regeneration slow and prone to exhausting memory, forcing workarounds (chunking the batch into smaller units with process restarts) just to complete a run.

Steps to reproduce

  1. On a default entity_usage install, leave the default track_enabled_plugins (all plugins enabled), including html_link, ckeditor_image, entity_embed, media_embed, and linkit.
  2. Create/save a node with a body field containing any HTML (a plain paragraph is enough; every enabled text-scanning plugin parses the whole field regardless of whether it matches anything).
  3. Add a temporary counter/breakpoint on Html::load() (Drupal\Component\Utility\Html::load) and save the node, or run drush entity-usage:recreate.
  4. Observe Html::load() is called once per enabled text-scanning plugin, per text field, per revision: 5 calls for the same field value instead of 1.
  5. On a site with a large number of revisions, observe memory growth and total regeneration time scale accordingly.

Proposed resolution

Add a small, bounded cache to the shared TextFieldEmbedBase class so a given field's text is parsed into a DOMDocument once and reused by every plugin that needs it, instead of each plugin parsing it independently. Since every plugin only reads the DOM via its own DOMXPath query (none of them mutate it), sharing the parsed document across plugins is safe and produces identical tracking results. This is a pure performance change with no behavioral impact.

We've prototyped and locally tested a patch that:

  • Adds a protected static array $parsedDomCache and a getParsedDom(string $text): \DOMDocument helper to TextFieldEmbedBase, keyed by md5($text) and capped at a small number of entries (4) to keep memory bounded across a regeneration run touching hundreds of thousands of distinct field values.
  • Updates HtmlLink, CkeditorImage, EntityEmbed, MediaEmbed, and LinkIt to call $this->getParsedDom($text) in parseEntitiesFromText() instead of Html::load($text) directly.

We verified (via reflection against the live plugin instances): the cache correctly reuses the same DOMDocument object across different plugin instances for identical field text, correctly misses (and reparses) for different text, stays capped at 4 entries under sustained use, and running trackUpdateOnCreation() against real content behaves identically to before the change.

Remaining tasks

  • Add automated test coverage: e.g. a kernel test asserting the number of Html::load()/DOM-parse calls drops when multiple text-scanning plugins are enabled against the same field (via a spy/counter), and that tracked results are unchanged.
  • Get maintainer feedback on the caching approach and the cache-size bound (4 was chosen empirically to cover a field being parsed both with and without HtmlLink's filter_url pre-processing, see its getTextFromField() override, plus one or two additional distinct text fields per entity).
  • Confirm no regression in trackOnEntityUpdate()/trackOnEntityDeletion() flows, and in EntityUsageTrackUrlUpdateInterface consumers.
  • Decide whether the cache should live on the base class (as prototyped) or be pulled out into an injected, swappable service.

User interface changes

None.

API changes

Adds a new protected method TextFieldEmbedBase::getParsedDom() and a new protected static property TextFieldEmbedBase::$parsedDomCache. These are internal implementation details on an abstract base class, not part of the module's public API contract, so this shouldn't be a breaking change, but it's worth flagging since third-party code extending TextFieldEmbedBase directly (rather than through the plugin system) could theoretically already define a member with either name.

Data model changes

None. The entity_usage table schema is unchanged, and the tracked usage records produced by a regeneration are identical before and after this change (same entities/references detected, same counts).

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

luongosb created an issue. See original summary.

luongosb’s picture

Version: 5.x-dev » 8.x-2.x-dev
Status: Active » Needs review
luongosb’s picture

Apologies for all of the updates on this issue. I'm still getting the hang of using Drupal's issue queue :)

I wasn't sure if I should create two seperate issues for the 5.x and the 2.x branch, so I attached both MRs to this issue.

Please let me know if you have any feedback! Thanks!

luongosb’s picture

Issue summary: View changes
luongosb’s picture

Issue summary: View changes