Problem/Motivation

ResourceObjectNormalizer::getNormalization() calls ResourceObjectNormalizationCacher::get() once per resource object.

A collection of N resources with M included resources can require N + M individual normalization-cache lookups, plus reads for redirects. With the default database cache backend, each read is a database query.

VariationCache::getMultiple() can be used to pull items in bulk with one backend read per redirect level. JSON:API does not currently use it.

Steps to reproduce

  1. Create 50 published articles.
  2. Bypass response caching while keeping the jsonapi_normalizations cache bin enabled.
  3. Request /jsonapi/node/article?page[limit]=50. Allow the request to complete, including the termination phase that caches the normalizations.
  4. Repeat the request while logging reads from the jsonapi_normalizations bin. Observe that resource normalizations are read individually, and there may be other reads needed to follow cache redirects.

Proposed resolution

Add ResourceObjectNormalizationCacher::prefetch(). It reads the normalizations of a list of resource objects with VariationCache::getMultiple() and provides a one-time hit/miss set for the current request. get() uses and removes a prefetched item if one exists; otherwise, it reads the cache as before.

Prefetched items are tied to the request that read them.

DataNormalizer::normalize() calls prefetch() before normalizing the items. This makes one batch for the primary data and one for the included data.

User interface changes

None.

API changes

None. (We add a method but this is @internal.)

Data model changes

None.

AI disclosure: I used AI to help prepare the IS and MR (Astra and Opus 5.5).

Issue fork drupal-3626255

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

ptmkenny created an issue. See original summary.

ptmkenny opened merge request !17265

deciphered’s picture

deciphered
Primary language English
Location Ballarat, Victoria, Australia
commented
Status: Active » Needs work

@ptmkenny I had an agent test this one against my site, since the other two would not touch its code paths (no fields disabled through JSON:API Extras, and its includes are entity_reference_revisions, which the relationship normalizer already loads per item before the include resolver runs).

Site: Drupal 11.4.6, Paragraphs, 39 nodes with 9 to 23 paragraphs each. Requests: individual nodes with include=field_content,field_content.field_media,..., and a 50-item collection with the same includes.

Backport blocker. As-is the MR fails on 11.x: every JSON:API request throws ServiceCircularReferenceException for jsonapi.serializer. DataNormalizer gains a constructor argument with an #[Autowire] attribute, but jsonapi.services.yml only has autowire: true in _defaults on main (from #3578374), so on 11.x the argument is never supplied. Adding this to serializer.normalizer.data.jsonapi fixes it:

    arguments: ['@jsonapi.normalization_cacher']

An 11.x backport needs that line or the autowire default. Everything below is the MR plus that line.

Query counts, from Database::startLog() on a fresh PHP process per request, page and dynamic page caches bypassed, normalizations already cached (before / after):

  • individual node, 18 included: 79 / 63 queries; normalization cache reads 20 / 4
  • individual node, 46 included: 107 / 63 queries; reads 48 / 4
  • collection, 39 nodes with 758 included: 1301 / 77 queries; reads 820 / 4

Responses were byte-identical. Counts repeat exactly between runs.

Timings, median of 40 HTTP requests, base and patched interleaved, opcache warm:

  • MariaDB over loopback TCP: the collection 154 ms to 97 ms (-37%), the 46-include node 23.1 ms to 21.3 ms (-8%), the 18-include node unchanged, jsonapi_menu_items with 38 links 32.7 ms to 27.8 ms (-15%).
  • SQLite: everything within noise, as expected when the saving is database round trips.

That is about 0.07 ms per removed read on loopback, so a database on another host gains more. The JSON:API kernel suite passes on 11.4.6 with the patch.

One review nit: prefetch() copies the two @todo lines about POST/PATCH and #3381898 from get() and set(), so the same note now appears three times in the class. A shared guard, or one note pointing at the others, would keep a future fix to one place.

Happy to run it against the site's production environment (MariaDB on a separate pod) once the 11.x form is settled.

ptmkenny’s picture

ptmkenny commented
Status: Needs work » Needs review

@Deciphered Thanks for the feedback! I have fixed the MR to work with Drupal 11.