Problem/Motivation
The links property is not always needed.
As such it is posing a proper overhead during generation and data transfers.
This started as JSON:API Extras issue, but due to complexity in how can that be overwritten from there, here we are :)...
Proposed resolution
Have a checkbox setting that will enable / disable links on a resource completely and so they are not generated at all.
In the specs it's stated what they should be with a proper format etc, but only if they are present.
In the specs it's not stated anywhere that they are a required property on all objects at all.
Where specified, a links member can be used to represent links. The value of each links member MUST be an object (a “links object”).
For example, an article’s comments relationship could specify a link that returns a collection of comment resource objects when retrieved through a GET request.
The optional links member within each resource object contains links related to the resource.
Remaining tasks
Patch, etc.
Benchmarks (payload size, generation time, used ram).
User interface changes
New checkbox to manage the functionality on a per resource level.
Maybe a single global one to manage the common default so we can spare developers clicking...
API changes
None.
Data model changes
New property expected on the configurable resource types.
Release notes snippet
TBD.
| Comment | File | Size | Author |
|---|---|---|---|
| #37 | jsonapi-issue-3039331-disable-links-37.patch | 6.34 KB | ab.shakir |
| #26 | drupal-stop-adding-links-in-jsonapi-3039331.patch | 5.22 KB | nils.destoop |
| #18 | jsonapi_extras-disable-links-3039331-18.patch | 6.77 KB | OlgaRabodzei |
Comments
Comment #2
ndobromirov commentedComment #3
ndobromirov commentedAfter a hacky PoC on this locally my total payload for a node (by ID) resource page dropped by about ~33%: 364 -> 246 kb (plain)
I have not done any benchmarks on the page generation times as of now. I will pass on the same hacky solution as a patch so anyone can at least test for himself.
What I've done is just commend-out everything that jsonapi module does for links generation in a non-error related context.
From what I saw, this might not be easy to do from jsonapi_extras perspective, as it does nothing in relation to the links generation and has little to no control for preventing that as well. For that reason only I am moving this up to jsonapi module's issue queue.
Comment #4
ndobromirov commentedHere are AB tests on the same page with and without the change.
No page cache.
No dynamic page cache.
No render cache.
Slowest request is 20% faster.
Mean request time is 10% faster.
Bytes transferred are 33% lower.
Patch is cumming shortly...
Running AB against apache on 81, as I have a varnish running on 80...
BEFORE:
AFTER:
Comment #5
ndobromirov commentedHere is the promised patch.
It differs a bit from my local tests, as latest dev and latest stable from 20th of February seems to be diverging a bit, but you can see the idea and the results should not be that different (unless I've missed some spot to comment out).
Comment #6
ndobromirov commentedComment #7
ndobromirov commentedSame patch (logically), smaller diff size.
Comment #8
e0ipsoThat is true, however there is strong consensus on the JSON:API team to require this in the Drupal implementation.
We should move this back to the JSON:API Extras issue queue and work on the implementation. I suspect the "new" links collector can be used for this. Just like
jsonapi_hypermediacan add links we should be able to remove them. JSON:API Extras will be the place to add the configuration (a single on/off toggle for all resource types, like the count feature).Comment #9
wim leersHow much difference is there with
gziporbrotlienabled?Exactly. For this use case, I think we can have a
jsonapi_hypomediamodule? 😆 (hypervshypo).Comment #10
ndobromirov commentedI am not saying to remove them. This is just the hacky PoC that I can live with for now (urgent stuff)...
But essentially this is just additional overhead. Why add something and then remove it :(. In the end it is going to be slower not having the links in...
It's ok to move it back to jsonapi_extras if it could be done from there.
@Wim, the 33% plain translate to ~16% gzipped. Dropped 34.3 -> 28.7 kb.
Comment #11
ndobromirov commentedComment #12
ndobromirov commentedAs agreed moving back to extras module.
Comment #13
wim leers#10: interesting, that's bigger than I thought. Thanks for letting us know!
Comment #14
ndobromirov commentedJust dumping this here as well, continuation of the previous patch. Ignore it.
Comment #15
OlgaRabodzei commentedHi!
I attach a patch, which provides possibility to enable / disable "links" objects per resource.
However a top level "links" remains a part of the response.
I think enable / disable option for optional top level elements like "links" and "jsonapi" should be in another configuration, which would be more general. Imho such configuration should be on `admin/config/services/jsonapi/extras` page with path prefix settings.
Comment #16
OlgaRabodzei commentedComment #17
ndobromirov commentedThere are 2 problems touched by this issue
1. PHP time spent generating URLs.
2. Response size in bytes, caused by excessive links in the response.
As it is, the patch from #15 solves only point 2, by adding some overhead to do the clean-up.
Comment #18
OlgaRabodzei commentedHere is an updated patch. I forgot to hide "links" checkbox on the configuration form, when a source is disabled.
Comment #19
OlgaRabodzei commentedAnd yes, Nikolay, I understand, that this patch doesn't make influence on the data generating performance.
But as it was mention in #8 comment, this is "a consensus on the JSON:API team to require this in the Drupal implementation".
So the only thing what we can do (without changing jsonapi module itself) is to play with the response view.
And I find the response less messy without all those links. Especially when it's read only endpoint.
Comment #20
e0ipsoI'm inclined towards allowing this. It seems a big enough difference to justify the downsides.
@ndobromirov is on point. I'd like to explore more before going for the second best solution (#18), which will provide a tangible win. Do you think we can decorate the link collector service to avoid generating the configured links? If not, do you think we can modify the collector to make this behavior possible? If we do that we can avoid generating the links in the first place.
Comment #21
e0ipsoI'm inclined towards allowing this. It seems a big enough difference to justify the downsides.
@ndobromirov is on point. I'd like to explore more before going for the second best solution (#18), which will provide a tangible win. Do you think we can decorate the link collector service to avoid generating the configured links? If not, do you think we can modify the collector to make this behavior possible? If we do that we can avoid generating the links in the first place.
Comment #22
ndobromirov commented@e0ipso for better or worse it's not a single place where the links collection class is used, and the links are generated right before instantiating the piece.
To have this implementable I think we need to have some things added / extended in JSON:API:
1. Factory method / service for link collections to be used through the code. That will create either a links collection or a NULL links collection based on the configuration. It will allow for more ways of creating the link collections, so we could accommodate this use-case. JSON:API addition.
2. Callback based construction and not a list of links one, so we can postpone link computation to a later moment, where we could respect configuration or anything.
3. Refactor existing code.
4. Consider API breaks...
Comment #23
ndobromirov commentedI will have to revive this for core's 8.7.11+ soon...
Comment #24
ndobromirov commentedThis can happen if we set
isLocatableon all resource types toFALSE.I am not saying we should, but if we expose this as resource type setting in JSON:API Extras it is 95%+ achievable.
Essentially preventing any links generation from JSON:API.
This will, in the most common case, prevent any links being generated and even if they are, it will be an empty set of links.
It will still be slower than the hack from #7 but a more cleaner solution overall.
Note that I am basing this on JSON:API from core 8.7.
If you know there is a better way in 8.8 I am open for suggestions, but at the moment I will work toward a patch with the setting on the resource types that will manage the isLocatable flag and from there integrate that into the configurable resource type altogether.
Comment #25
bbralaAny progress on this? :)
Comment #26
nils.destoop commentedI re-rolled the patch in #7 for 9.2
Comment #27
bradjones1I think actually this is better handled in core, if we have @bbrala's attention on this issue. See related.
Comment #28
ro-no-lo commentedAre there still any arguments left to have a switch in core to remove all links and their respective generation, if the use case do not need them? Even if the Drupal Json:Api Core team has a strong standing for providing these links in the output, if the individual website services don't need them, why generating and outputting them against their will and need? It seems like we "fight" over an optional JSON:API Refernece feature which will require only a few if conditions here and there.
I stumbled across this 4 years old issue, because I also want to remove them for our vue app. https://www.drupal.org/project/jsonapi_extras/issues/3311150
Comment #29
ptmkenny commented@ro-no-lo My understanding is that the reason the links are included without any option to remove them is because that is what is required by the formal JSON:API specification, and the core module aims to provide 100% support for the specification. (See this comment in response to my question about removing links for NULL entity references.)
I understand the case for forcing the links always (if you say the module adds support for a specification, it should follow that specification), but at the same time, your use case and mine could really benefit from slimming down the responses by removing the links, and I am sure there are other sites in the same boat. So even if it violates the specification, I think this would be a very useful option.
Comment #30
e0ipsoI think this feature is often an anti-pattern. I am not very supportive of promoting this practice. Lack of usage of hypermedia capabilities is one of the biggest factors in maintenance problems in decoupled applications.
If your server is using a compression algorithm the difference in the response length should be negligible. If not, that will be a way bigger win across the board than removing some links in JSON:API responses.
That being said, I will defer final decision to @bbrala.
Comment #31
e0ipsoAlso note that the initial report was created before we improved internal caching of normalizations, so the generation of links should not be as big of a problem nowadays. Can anyone corroborate this?
Comment #32
wim leersI agree with @e0ipso.
If you want to modify all JSON:API responses to not contain any links, that's easy enough to achieve:
jsonapi_linklesscontrib moduleKernelEvents::RESPONSEevent subscriber that runs afterjsonapi.resource_response.subscriber(which has priority 128), with a priority lower than Dynamic Page Cache's (100), to ensure the results are cached by Dynamic Page Cache — so any priority in the [101, 127] range.linksfrom the response in that event subscriber, if the response is a JSON:API responseComment #33
ro-no-lo commented@ptmkenny: The jsonapi.org states that the resource links are optional. https://jsonapi.org/format/#document-resource-object-links
@e0ipso: currently we develop a vue3 app and these modern app come with a lot of packt vuex stores. With more and more entities loaded into the browser system everything grows. I do not have enough expertise to know what will happen, when every call to the json:api and lists of entities come with a 30% overload of unused data. We may be not experienced enough to build such a hypermedia usage into the system, but thats were we are currently. My concern is, that the memory usage is growing for processing the JSON responses from the server, even though it could be smaller. Time will tell.
@wim-leers: Thanks for the tipp. I will consider it, when we run into memory problems.
Comment #34
e0ipsoI think this may not be a problem as per #31.
I understand that, and I empathise with the situation. However, as a maintainer this is not something I want to prioritize in this module. But, again, I will follow what @bbrala decides since he is carrying the baton of this module.
Comment #35
ro-no-lo commentedFor everybody interested. I managed to put a module together with the hints Wim Leers gave me. https://www.drupal.org/project/jsonapi_links If you find stuff you could optimize, do so and create a merge request.
Comment #36
allahnoor turab commented26 and 18 are not applying to drupal version 9.5 for me
Comment #37
ab.shakir commentedRe-roll of #14 for D9.5.8.
Comment #38
e0ipsoThanks for all the contributions! It sounds like ro-no-lo provided a successful solution to this in #35.
Closing.