Closed (fixed)
Project:
JSON:API
Version:
8.x-2.x-dev
Component:
Code
Priority:
Normal
Category:
Task
Assigned:
Unassigned
Issue tags:
Reporter:
Created:
23 Jul 2018 at 15:40 UTC
Updated:
11 Oct 2018 at 17:29 UTC
Jump to comment: Most recent, Most recent file
Comments
Comment #2
gabesulliceComment #4
wim leersI'm less convinced by this change actually. Can you explain the rationale?
Comment #5
gabesulliceThe rationale is simply that it's a necessary step to get rid of the request handler. I should have already put this in the [META] issue, this issue comes with the understanding that deserialization will later be moved to a route enhancer.
Why?
Today, we have a small bit of overhead on every request.. the first line of
RequestHandler::handleis$parsed_entity = $this->deserialize($request, $resource_type);. That runs even for GET requests (at least as long as it's not already cached). Moving to specific methods means we can avoid that small step.After that's done and arguments are being dynamically resolved, we'll be able to move this back out of EntityResource into a route enhancer so it'll run for only methods with a
$parsed_entityargument. We can't do that now without adding complexity to theRequestHandler.Comment #6
wim leersmeans that
::deserialize()returns early forGETrequests.Ah, that would make it worthwhile indeed!
IOW: this is a step on the path towards moving the mapping of a request body (through deserialization) to a controller parameter. Correct?
Comment #7
gabesulliceYep, and this is somewhat of a workaround for that fact that we don't currently have a way to call
deserializejust for the routes that need it.Exactly.
Comment #8
gabesulliceI was expecting #2987606: Remove config mutation tests from EntityResourceTest to be committed fairly easily, so I let this issue depend on it. That's not entirely necessary though.
So instead of postponing this issue on that one, I rerolled this patch to fix the config related tests rather than just assume that they're gone.
Comment #10
gabesulliceThis should pass 🤞
There are two interdiffs, one shows the change to make the tests pass, the other is code standard fixes. Both are part of the patch.
Comment #11
wim leersDo we really need this to be public?
I think this test is the only reason we make it public? If so, let's just define a test-only service in the test's
setUp()that is a public alias of this private service. That will achieve the same!Nice bugfix!
Shouldn't this be moved to
jsonapi_extras?Why this change? (It seems to be removing valuable test coverage?)
Why this change? (It seems to be removing valuable test coverage?)
Comment #12
gabesullice1. Instead of making an alias, I just instantiated it in the setUp method.
2. :)
3. I added the test back and marked it skipped with a note to move it in #2977659: Spec Compliance: POST|PATCH|DELETE on relationships should respect arity rules.
4+5. Added this coverage back in. #2977659: Spec Compliance: POST|PATCH|DELETE on relationships should respect arity rules will also add much more comprehensive coverage for all these scenarios (including cache info, status codes, etc).
Comment #13
gabesulliceFiles actually attached this time.
Comment #14
wim leers👍 Confirmed these are identical before & after!
You restored this test coverage, but it looks *very* different now. Is that truly necessary? It makes this patch much harder to review, and the interdiff twice as large.
I've just applied this patch plus #2987609-21: Rename the entity parameter from the entity type ID to 'entity' for all routes plus #2987610-10: Remove RequestHandler class and service and add EntityResource methods to each route definition, to see the end result.
20 files changed, 431 insertions(+), 448 deletions(-)is not as exciting a diffstat as I'd hoped … 😞How can a route enhancer deserialize a request body? AFAIK this is not possible?
Comment #15
gabesulliceJust responding to the last bit ATM...
See
JsonApiParamEnhancer. We already access the request's query parameters and denormalize them into aFilterobject.We would pretty much do the same as we do there. I.e., access the request, get its content, then add the parsed value as a default route argument. That default would be treated as a parameter to
EntityResource::{method}()as resolved by the arguments resolver.After thinking about it more though, I think we may be able to do that today, not tomorrow. Previously, I didn't see a way to take the route enhancer approach while the
RequestHandlerstill existed... and I didn't know how to get rid ofRequestHandlerwhile it still did deserialization. Hence I wanted to do it in 3 steps. Move deserialization, remove the request handler, then put deserialization in a route enhancer. BUT, I have an idea that I'd like to try out that will get rid of this step. Stand by!Comment #16
gabesulliceOkay, here's the approach as discussed in #15. That cut the patch size in half! And we managed to make more deletions than insertions! Breaking it down by src/test directories, it looks even better:
So, what happened?
I used a route enhancer to deserialize the entity into a default parameter and then accessed that deserialized value in the
RequestHandlerbefore passing it toEntityResource.That's just what we need to do in order to set up #2987610: Remove RequestHandler class and service and add EntityResource methods to each route definition.
With this approach we barely need to touch
EntityResourceat all. The only thing I did was make it a bit more consistent because using route info already available to better understand when the body actually needs to be deserialized and when it doesn't.Unfortunately, I had to make
jsonapi.serializer_do_not_use_removal_imminentpublic... I spent way too long trying to figure that one out and finally just gave in. I can't figure out how to alias it in a BrowserTest because$this->container->setAliasisn't a available. I couldn't simply instantiate it myself because we need the serializer to be injected with all the normalizers and I can't get those tagged services.This public/private thing seems to be a recurring issue. I think we need to take some time to figure out a good solution for this as a general solution rather than struggling with this over and over.
Finally, the only failing tests I know of ATM, are the unit tests for
JsonApiParamEnhancer, again, because of the private service issue. I honestly think we can remove that test class though, but I wanted to get others' opinion first.Comment #18
gabesulliceCS fixes.
Comment #19
gabesulliceAnd here it is with
tests/src/Unit/Routing/JsonApiParamEnhancerTest.phpremoved if no one objects to removing it.Comment #20
wim leers#15: Ahhhh, of course! Makes sense.
#16: Woah! 😮 Now that is an interdiff :D It does look like some other patch got entangled in there though.
#19: I first wanted to bring back the test coverage. But then I realized that the existing test coverage was literally just verifying that a specific normalizer would be called, and the logic in HEAD for
JsonApiParamEnhancerwas in fact already hardcoding a specific normalizer for a specific purpose. So the test coverage was not really testing anything. I agree that this test coverage has served its purpose (dating back to JSON API's early days, added in May 2016, commit2a2b593!)I wonder if this has a performance impact.
In HEAD, we're instantiating 3 normalizer services during routing.
With this, we're instantiating the entire JSON API serializer and hence all JSON API normalizers … but also core's serializer and all of its normalizers.
From 3 services in the critical routing path to many dozens.
That means this approach definitely needs profiling.
🙃😵
Comment #21
gabesullice1. Rerolled.
2. Hm. I'll need some guidance on how to do that.
3. I don't understand those emojis :P did you want something to change, or are you just remarking on its insanity?
Comment #22
gabesullice#2994700: Remove unused argument from the link manager should address the #20.3 craziness.
Comment #23
gabesullice#2994700: Remove unused argument from the link manager landed. Rerolled.
Comment #24
gabesulliceMade the title more accurate.
Comment #25
gabesulliceAgain.
Comment #26
gabesulliceThis blocks #2987610: Remove RequestHandler class and service and add EntityResource methods to each route definition.
Comment #27
gabesulliceRerolled.
Comment #28
gabesullice#20.2 pointed out that the
JsonApiParamEnhanceris instantiating normalizers for every routing invocation. This patch might exacerbate that because it instead instantiates the entire serializer and all normalizers.Rather than let this get blocked on that case and profiling, let's just lazy load the serializer under all circumstances.
Comment #29
wim leersPragmatism FTW! 👍
I'd like to see this change removed from the patch. Is that still not possible?
What is again the problem with keeping these as-is?
Or, put differently, why are these even
normalizerservices at all?Ah, this means it's not possible.
Comment #30
gabesullice2. At first, it was because we also needed the serializer for deserializing the incoming entity or resource identifiers. Therefore it seemed excessive to individually inject every one of those. Later, it's because it turns out that in many instances, we don't need them at all. Since this is on the critical path, perhaps it's better to avoid loading anything. OTOH, maybe that a micro-optimization.
They're normalizers because we're taking a particular wire encoding (the query parameter string), decoding it into an array, and producing a non-scalar object. Which is exactly what the serialization system is designed to do 🙂 (decode -> denormalize or normalize -> encode).
This was set to "Needs work", but I don't see any word to be done. Am I missing something?
Comment #32
wim leersIt was NW for #29.1, and kinda for #29.2.
#30.2: yes, you could theoretically argue that way that this is denormalization for the reasons you cite. But … only barely so. According to the same line of reasoning,
\Drupal\Core\ParamConverter\ParamConverterInterface::convert()should be removed in favor of a denormalizer too. Because really, this is mapping information in a URL to a value object. The opposite direction (normalization) never occurs. I think it's pretty clear this is not using the Symfony serialization system fore the intended purpose.Another reason you cite is to not load anything unless they're actually needed, but … that's only because we're using the serialization system at all. Usually, I'd argue that this means that new things can be added in the future (in this case: new parameters). But that's not even true, since
\Drupal\jsonapi\Routing\JsonApiParamEnhancer::enhancerequires a mapping from URL query arg name to denormalization class to be hardcoded anyway!If we'd remove these normalizer services, we'd have less weirdness, and compared to #28, we'd be able to not inject normalizer services nor pass the entire container. Passing the entire container is always frowned upon.
Additionally, all this logic only makes sense (only runs) for JSON API collection resources anyway. Just grep for
_route_paramsand_json_api_params. So rather than implementing\Drupal\Core\Routing\EnhancerInterface, which is executed at runtime (because the signature ispublic function enhance(array $defaults, Request $request);), it'd be much more logical to run this in\Drupal\jsonapi\Controller\EntityResource::getCollection()instead. That's the only place where it's being used. And hence it's the only place that should be slowed down by this.…
But doing all that would be a much bigger refactoring. It would remove the bulk of
JsonApiParamEnhancer::enhance(). It'd remove what that was originally designed for (i.e. "the JSON API params", meaningfilter,sortandpage), but the deserialization of the request body would still be left to be done. So the serialzier would still need to be injected.So, rather than insisting things be done even better, I think this patch is already better than what's in HEAD. There's a net negative diffstat. And it directly paves the path for #2987610: Remove RequestHandler class and service and add EntityResource methods to each route definition, which will result in the
RequestHandlerclass being deleted. Core REST will be handling its request body deserialization in itsRequestHandlerclass, JSON API will be handling it in a param enhancer. Neither is perfect.My chief concern is that this may get in the way of #2958554: Allow creation of file entities from binary data via JSON API requests, which is IMHO far more important than this because it brings an important new capability to JSON API. In core REST, this required changes to
RequestHandler. But #2958554 is doing stuff with an alternativeRequestHandler, just like core REST. So I think that before this can be committed, we need to land #2958554: Allow creation of file entities from binary data via JSON API requests (ideal case) or we need to be very certain that removing JSON API'sRequestHandlerwon't get in the way. Hence keeping at .Comment #33
gabesulliceI'm very certain that it won't.
#2987610: Remove RequestHandler class and service and add EntityResource methods to each route definition removes the request handler entirely. It is replaced by route-specific calls to a service and specific methods.
#2958554: Allow creation of file entities from binary data via JSON API requests lives in a submodule and has its own
RequestHandlerwhich is just a service+method. It does not extend the JSON API request handler. Having been a guiding hand in the file upload issue, I can say that a clean separation of concerns was at the forefront of my mind precisely for this reason. I knew I wanted to remove the request handler.And I know that I want to remove the request handler for cases like this one. Being able to "weave" in alternate JSON API routes because we don't have a hardcoded bottleneck in the RequestHandler was/is one design goal of #2953346: Define related/relationship routes per field, not dynamically (with route parameters that need validating) and #2987610: Remove RequestHandler class and service and add EntityResource methods to each route definition.
In fact, I have a chat record with @malik.kotob where we discussing this in relation to file uploads:
Given that the file uploads patch operates with a completely distinct handler and that it uses a subrequest to return its responses. I think we can safely proceed.
Comment #34
wim leersPer #28.
Comment #35
wim leersI wrote in #32:
@gabesullice wrote in #33:
But because it's not trivial to see, I wanted to answer this by proving it with passing tests: #2958554-80: Allow creation of file entities from binary data via JSON API requests is #2958554 + this patch + #2987610: Remove RequestHandler class and service and add EntityResource methods to each route definition. It passes tests. Therefore there cannot be any doubt in anybody's mind that this is safe to do. Hence: RTBC, and committing :)
Comment #37
wim leers#2987610: Remove RequestHandler class and service and add EntityResource methods to each route definition unblocked!
Comment #38
gabesullice🎉