Problem/Motivation
The ResourceIdentifier class that is used to add all entity references to the relationship is currently using an entity load to add all relationship entries to the jsonapi output.
This can lead to a huge set of queries being executed for collection requests, as all references of the nodes are loaded (paragraphs, users, terms, node references, ...)
In my example this leaded to 8372 for cold cache hits, because the client had a total of 1k paragraphs that were used on articles in the result set.
Using entity load for the relationships is heavy, as my json api requests are not requesting info about those relations. The paragraphs are only requested on the detail page of a node. The users are also loaded, while no user info is displayed in the responses.
SELECT "t".* FROM "user__field_password_expiration" "t" WHERE ("entity_id" IN (:db_condition_placeholder_0)) AND ("deleted" = :db_condition_placeholder_1) AND ("langcode" IN (:db_condition_placeholder_2, :db_condition_placeholder_3, :db_condition_placeholder_4)) ORDER BY "delta" ASC
SELECT "t".* FROM "paragraph__field_text_body" "t" WHERE ("entity_id" IN (:db_condition_placeholder_0)) AND ("deleted" = :db_condition_placeholder_1) AND ("langcode" IN (:db_condition_placeholder_2, :db_condition_placeholder_3, :db_condition_placeholder_4)) ORDER BY "delta" ASC
| Comment | File | Size | Author |
|---|---|---|---|
| #12 | drupal-no_entity_load_for_relationships-3255840-12.patch | 2.49 KB | leon kessler |
| #7 | overview-profiling.png | 229.81 KB | nils.destoop |
| #7 | patch-only.png | 418.54 KB | nils.destoop |
| #5 | 3255840-5.patch | 2.49 KB | ranjith_kumar_k_u |
| #2 | drupal-no_entity_load_for_relationships-3251888-18.patch | 2.44 KB | nils.destoop |
Comments
Comment #2
nils.destoop commentedI'm using this patch to convert all the entity load to 1 query to the base table. The resource identifier only needs a uuid, bundle and entity type.
Next to that, I also implemented the hook_entity_field_access in my custom module, to set view access to false for entity references (like paragraphs) on json api collection requests that are not needed on lists in my project.
As extra tweak I used a custom version of the patch in https://www.drupal.org/project/jsonapi_extras/issues/3039331#comment-143...
Before changes: 8372 queries
After applying this patch + implementing access rules on the fields: 1486 queries
After also applying the patch for link fields: 1116 queries
Comment #3
nils.destoop commentedComment #4
cilefen commentedCan you measure the execution time of this proposal vs the main branch and report that here?
Comment #5
ranjith_kumar_k_u commentedComment #7
nils.destoop commentedOverview of profilings:
Summaries:

https://www.drupal.org/files/issues/2022-01-24/overview-profiling.png
Difference between with patch and without patch:

https://www.drupal.org/files/issues/2022-01-24/patch-only.png
Comment #8
bbralaThank you for the profiling. Seems like there is some real performance to be had. I wonder though, would disabling the field (see https://www.drupal.org/docs/core-modules-and-themes/core-modules/jsonapi...) also work in reducing those calls?
Comment #9
nils.destoop commentedNo it didn't. I use json api extras to disable some of those fields. But they still were loaded and removed afterwards. Implementing the field_access hook makes sure the related normalizer is not activated:
https://git.drupalcode.org/project/drupal/-/blob/9.4.x/core/modules/json...
Comment #10
yang_yi_cn commented@nils.destoop can you explain how can I implement my own hook_field_access to speed it up?
I think I sometimes get hit by this issue too and it takes forever to load certain resources, which makes my app not usable.
Comment #11
nils.destoop commented@yang_yi_cn my version:
It disables the social image relation and the paragraphs relation on collection requests (except for 1 content type, special case for the app).
For all other pages it also disables some basic fields.
Comment #12
leon kessler commented@nils.destoop thanks for posting all your work, including profiling info.
We're also experiencing slow response times on cold cache requests.
e.g. hitting
/jsonapi/node/pagetakes around 6 seconds. When adding something like?include=field_referenceit goes up to around 9 seconds.Not quite as bad as your situation with paragraphs. Our site doesn't use paragraphs, but is quite heavy on taxonomy. There are around 7 or 8 taxonomy reference fields on each content type.
Our issue we have lots of cache tag invalidations occurring, due to taxonomy terms often being updated (so large chunks of content get cache wiped out in one go). So having 6-9 second response times on these requests is not ideal.
When trying your patch and example of
hook_entity_field_access()the amount of db queries was reduced, from 1731 to 1072. However response times are not noticeably improved, so I'll continue investigating. Possibly there's not a huge amount we can do, and may turn to using cache warmers and some slight tweaking to how cache invalidation works.Anyway, in regards to your patch. I've re-rolled it to apply cleanly to 9.3.x
I also encountered the following error:
This occurred on
/jsonapi/node/page. It occurs when it tries to loadtypefield (i.e. the node type). I addedtypeto the list of$excludedFieldsinhook_entity_field_accessand this fixes the issue. Although only for anonymous users.