Normalizers should be as straightforward and simple as possible.
Directly accessing the request object shouldn't be a part of their responsibilities as it is here:
/**
* Expand the context information based on the current request context.
*
* @param \Symfony\Component\HttpFoundation\Request $request
* The request to get the URL params from to expand the context.
* @param \Drupal\jsonapi\ResourceType\ResourceType $resource_type
* The resource type to translate to internal fields.
*
* @return array
* The expanded context.
*/
protected function expandContext(Request $request, ResourceType $resource_type) {
// Translate ALL the includes from the public field names to the internal.
$includes = array_filter(explode(',', $request->query->get('include')));
// The primary resource type for 'related' routes is different than the
// primary resource type of individual and relationship routes and is
// determined by the relationship field name.
$related = $request->get('_on_relationship') ? FALSE : $request->get('related');
$public_includes = array_map(function ($include) use ($resource_type, $related) {
$trimmed = trim($include);
// If the request is a related route, prefix the path with the related
// field name so that the path can be resolved from the base resource
// type. Then, remove it after the path is resolved.
$path_parts = explode('.', $related ? "{$related}.{$trimmed}" : $trimmed);
return array_map(function ($resolved) use ($related) {
return implode('.', $related ? array_slice($resolved, 1) : $resolved);
}, FieldResolver::resolveInternalIncludePath($resource_type, $path_parts));
}, $includes);
// Flatten the resolved possible include paths.
$public_includes = array_reduce($public_includes, 'array_merge', []);
// Build the expanded context.
$context = [
'account' => NULL,
'sparse_fieldset' => NULL,
'resource_type' => NULL,
'include' => $public_includes,
'expanded' => TRUE,
];
if ($request->query->get('fields')) {
$context['sparse_fieldset'] = array_map(function ($item) {
return explode(',', $item);
}, $request->query->get('fields'));
}
return $context;
}
IOW, we should be "expanding" that context prior to (de)normalization.
Comments
Comment #2
wim leersAgreed.
Super hacky first attempt to see what breaks. I won't be working on this anymore today, so feel free to continue this!
Comment #3
wim leersI'm of course not proposing to keep the logic here and keep it as a public method … that's just to make the patch as small as possible in a first attempt!
Comment #5
wim leersMost important problem:
Comment #6
wim leersComment #8
wim leersThis is the problem with kernel tests.that are sort of integration tests but should be unit tests.
Comment #9
wim leersGreen!
Now let's address this:
Comment #11
wim leers98 fails.
Because
in
\Drupal\jsonapi\Normalizer\Value\JsonApiDocumentTopLevelNormalizerValue::rasterizeValue().Thoughts?
For now, undoing #9. (i.e. going back to #8)
Comment #12
wim leersMeanwhile, moved the context generation logic from
JsonApiDocumentTopLevelNormalizerto\Drupal\jsonapi\EventSubscriber\ResourceResponseSubscriber::generateContext().This now needs feedback.
Comment #13
gabesullice😍
"Generates a top-level JSON API normalization context."
"The request from which the context can be derived."
I don't think we can call this done until this is removed.Ofc, that's what you're asking about in #11... see below.I think this is just the kind of information that should be carried by the
JsonApiDocumentTopLevelobject. I'd add a$linksargument to its constructor which receives an array ofUrlobjects keyed by link relation type.In that constructor:
assert(empty(array_diff(array_keys[$links], ['self', 'first', 'next', 'prev', 'last', 'related'])))Then, do all that work with the link manager in the controller rather than in the normalization process.
This will make it easier to capture that pesky cache data too!
Comment #14
wim leersComment #15
wim leersThis is not 100% done, but does handle the majority of the structural changes. Looking for a review of those changes.
Not yet done:
stringtoUrlwas not made yet; that requires extensive changes toLinkManager.Comment #17
gabesulliceInitial review below. I made a few nits that I know you said were not yet done, but /shrug :P
Overall, I love the direction of this patch!
What kind of changes? Perhaps we should make a prerequisite issue for it? My dream would to see LinkManager simply removed, or made into an internal class with public static helper methods, or a really solid API for adding custom links to various levels of the document introduced. Or some combination of those things.
I'd like to see the self link be explicit, unless that would explode the scope of this patch.
This shows that the request is no longer added part of the normalization process.
Nit: I would call these "pagination_links"
I like that this extracts the link manager from the normalization process in at least one case. It always felt a little like the old
CurrentContextclass, in that it weaves its way through everything."The URLs to which the top-level document should link."
Perhaps we should document the allowed link relations in the docblock.
s/document/top
This will need to be adjusted to account for #2987206: Refactor `getEntityAndAccess` to return cacheable objects with access cacheability rather than an entity/access pair.
Missing a comment.
This is a much nicer signature for a value object.
Comment #18
wim leersFirst, rebased, because #2942549: Spec Compliance: JSON API allows POSTing relationship fields in 'attributes' rather than in 'relationships' and #2977669: Spec Compliance: some entity types have an "id", "type" or "uuid" field, this violates the spec made this no longer apply. And #2987603: Make resource type an explicit parameter to each EntityResource method made this fail tests.
Next: address #17.
Comment #20
wim leers🎉
It totally should be. Let's mark itLinkManageris already an@internalclass.@internalhere?Those extensive changes are: A) change the return signature of every single method on
LinkManager, B) makeLinkManager::getRequestLink()no longer justreturn $request->getUri()if there is no query.selflink?Comment #21
wim leersI missed a spot where #2987603: Make resource type an explicit parameter to each EntityResource method forces me to make changes here.
Comment #23
wim leersNit: retain/restore a rename that #2977669: Spec Compliance: some entity types have an "id", "type" or "uuid" field, this violates the spec made after this patch already existed.
Comment #26
gabesulliceThat's a first step in the right direction :)
What I was trying to say is I don't think it the LinkManager should have its own state, i.e. if every method is public static, its methods exist only to avoid code duplication. IOW, this would ensure that it's no longer a dependency woven through everything. It would become a simple set of reusable functions. I don't think that should happen here.
OTOH, I've also it at the back of my mind that JSON API needs a public API for customizable HATEOAS links. My last remark and this need are related, but not compatible. I just want it to be on your mind too.
Exactly. In general, I'd like to see links explicitly passed to
JsonApiDocumentTopLeveland to each value object that has links, rather than having value objects generating their own links.3 & 5. WFM!
Comment #27
wim leersThis fixes all but one failure. That last failure is tricky: it's the
meta.countthat is missing. It used to be handled in\Drupal\jsonapi\Normalizer\Value\JsonApiDocumentTopLevelNormalizerValue::rasterizeValue(). I accidentally removed it (yay for tests!), but … to be able to do that, we'd either need to:$context['total_count'], which is exactly what we don't wantResourceTypeto theJsonApiDocumentTopLevelNormalizerValueconstructor, so that we can do$collection->getTotalCount(), but that'd mean we're gathering the count in two places: once inJsonApiDocumentTopLevelNormalizerValue::rasterizeValue()during rasterization, plus once in\Drupal\jsonapi\Controller\EntityResource::respondWithCollection()during link generation.I don't like either.
Ideas?
Comment #28
wim leers#26:
JsonApiDocumentTopLevelNormalizerValueused to generate its ownselflink, and that now lives in the controller … based on what you said here, that'd be the opposite of what you want?It seems impossible to combine the goal of this issue ("normalizers unaware of
Requestobject") with the intent to have normalizer value objects generate their own links? Or what am I missing?Comment #30
gabesulliceI think you understood the reverse of what I wrote. I do not want any value objects to generate their own links (or call the LinkManager).
Will respond to #27 soon. I'll have to stare at it a bit.
Comment #31
wim leersYou're right, oops 😇😁😭
Re-reading isn't enough for me, I apparently need re-re-reading. 👴
Comment #32
wim leersFixed CS violations.
Comment #34
gabesulliceRe: #27
What about something like this? Can't it be an explicit argument to the value object? Am I missing something?
Comment #35
wim leersIt totally can. But it's not very elegant, is it?
Then I'd much rather have not
$total_count = FALSEin the constructor, but$meta = [], which we'd deep-merge. That'd allow any controller to pass additional meta-stuff; rather than it being specific for collection counts. Might be useful in the future. Is less tightly coupled.Comment #36
gabesulliceIt's not very elegant, but it's an improvement :P
$meta ...in #35 is a good suggestion.Comment #37
wim leersSo what about this?
Comment #38
gabesulliceCouple nits, and a q. After that, RTBC I think.
Why not inject the request?
Nit: Can we just add this to the array conditionally, rather than overloading
total_count?Nit: let's use $resource_type->includeCount() instead of checking for FALSE.
Comment #39
wim leersComment #41
wim leersI think you actually meant this in #38.1.
Comment #42
gabesulliceAwesome :)
Comment #44
wim leers#2985426: Spec compliance: `related` routes should return 200, not 403, if field access is allowed but the related resources are forbidden and #2991389: Test coverage: relationship response cacheabiliity conflicted with this.
Comment #45
wim leersRebased.
Exciting: the two lengthy
@todos inJsonApiDocumentTopLevelare now condensed to a single much shorter one!Tricky: #2949807: Spec Compliance: Error responses are missing the `jsonapi` top-level member started using
JsonApiDocumentTopLevelNormalizerValue's$contextparameter to indicate whether this was for an "errors document" or not … but the patch here was removing the$contextparameter! For now, brought it back.Until #2965056: Support `include` parameter on relationship routes (turns out it accidentally works on cold caches: fix + test this) lands, this will be necessary. Actually, it's worse … because the whole point of this issue is to remove access to theI worked around this by replacingRequestobject during normalization. But #2991389: Test coverage: relationship response cacheabiliity made that impossible :( At least until #2965056: Support `include` parameter on relationship routes (turns out it accidentally works on cold caches: fix + test this) lands.$context['request']with\Drupal::requestStack()->getCurrentRequest().This was one super difficult rebase 😵 Really hoping this passes tests! 🤞
Comment #46
gabesulliceThis was my fault.
Can't we just add aNo, because then we'd have to add the$context['on_relationship'] === TRUEingenerateContext?$contextparam back.Comment #47
wim leers#46: exactly! That's what I first tried to do, but then I realized this was better/more elegant, since it only exists temporarily anyway. See the
<del>part of #45.Also: HOLY SHIT this is green from the first time!!!!!!! 🎉🎉🎉🎉🎉
Comment #48
gabesullicePerhaps we can add these cache contexts somewhere else then? E.g. in
EntityResource::getCollection()AND HECK YEAH :D
Comment #49
wim leersOnce this is in, we'll also be able to make
\Drupal\jsonapi\Controller\EntryPointmuch nicer! 😀Looking forward to an RTBC on this one :)
Comment #50
e0ipsoI will be taking a look at this shortly. Thanks for all the work. Everything I saw in a quick scan made sense to me so far.
Comment #51
wim leersThanks, @e0ipso!
I wanted to address #48, but that led me to realize that this leads me to be quite far out-of-scope. Created #2992673: Set collection-specific query parameter cache contexts on collection responses instead of all responses for this.
Comment #52
gabesulliceBump.
Comment #53
e0ipsoI know I've said this in the past already, but I feel that these refactors in the name of maintainability have subjective value. One can see them as improvements, and another as a step back. Some of the brightest minds of the Drupal community have been puzzled by the inner workings of the module due to the lack of a cohesive execution thread to follow. In this particular case I think this issue is a step in the right direction.
Some generic comments.
Why do we need to separate links out, and why a polymorphic argument?
I'm unsure this level of babysitting is necessary.
Hmmm, this seems like an abuse of the parameter.
Again, I'm not sure I understand what's the value of all these assertions.
Comment #54
gabesullice❤️
1. IIRC, because value objects probably shouldn't be generating their own links. Also, doing so removes the need to pass in the
LinkManagerwhich depends on the request object. I agree with your confusion about acceptingstringor aUrlobjects. I'll investigate.2. I wouldn't call this particular instance "babysitting." This guarantees that the response subscriber is getting an upcasted value, not a string. Given the complexity of the routing system, I'm happy to have this assert there so that we can quickly pinpoint when/if that ever breaks.
3. I agree. In a better language we'd make this an explicit argument using an enum. I did my best to replicate that in the attached patch.
4. The first was my suggestion in #13 and I remember @Wim Leers was actually not entirely sold either. Since then, I've actually come to think about link object keys differently. Therefore, I've removed the first assertion because I agree that it's not helpful. WRT to the second assertion, I guess it comes down to the same arguments that exist between type safe and untyped languages. My preference is on the latter, so I appreciate these kinds of type assertions.
Comment #55
gabesulliceRe: #54.1 I removed the Url stuff. I imagine @Wim Leers was prepping to better handle link cacheability, but that can happen in a separate issue.
Comment #57
gabesullice2 CS violations and forgot to update a unit test.
IMHO, this is ready.
Comment #58
e0ipsoI use git hooks to make sure DCS don't go to d.o https://asciinema.org/a/pqa3uRQwf9Uq6d4I7iaQf1WxZ
If you are interested on that I can share the setup for contrib development.
Comment #59
gabesulliceI lied. This gets rid of some outdated comments since #55.
Comment #61
e0ipsoMerged.
Comment #62
gabesullice🎉
Comment #63
wim leers+1
🎉
Comment #64
wim leersThis unblocked #2984911: Remove access to the Request object in the normalization process.
Comment #65
wim leersQuoting myself in #49:
Issue: #3000622: Improve EntryPoint::index() now that JsonApiDocumentTopLevel is more capable.