Problem/Motivation

For a collection endpoint which is trying to load 50 nodes with ~30 field(base, configurable and computed) onn a single page request, there are ~6000 calls to \Symfony\Component\Serializer\Serializer::getNormalizer() though the time spent is not that signficant but this number is way too big.

Proposed resolution

TBD

Remaining tasks

TBD

User interface changes

TBD

API changes

TBD

Data model changes

Comments

jibran created an issue. See original summary.

jibran’s picture

Some notes from slack:

neclimdul : @jibran the summaries look cool but ...who are the callers? without the graph all the context is kinda lost

I'll add that later today.

gabesullice: @jibran, you said you had one request that was 15 seconds. Is that right? Is the bulk of that in normalization?

I shared a screenshot in slack. I'll add it here as well.

mglaman: @jibran FYI I dug into this before, and it's just poor performance inside of core's serialization process IIRC
_or_ symfony/serializer itself

such as checking the normalizer for each data type that it encounters.

Apparently, this has been fixed in symfony 4.

wim leers’s picture

Title: Too many calls to \Symfony\Component\Serializer\Serializer::getNormalizer() » [upstream] Too many calls to \Symfony\Component\Serializer\Serializer::getNormalizer()
Category: Bug report » Task
Status: Active » Closed (works as designed)
Issue tags: +Performance, +API-First Initiative, +Needs upstream bugfix

Per

such as checking the normalizer for each data type that it encounters.

this is definitely not a bug. In fact, there are zero ->getNormalizer( calls in the JSON API code base.

wim leers’s picture

jibran’s picture

Let me add the caller details as @neclimdul requested. I personally think there are some static caching, event prioritization and edge cases on which we can make some improvements.

FWIW, getNormalizer is a private method so no one can use it outside the \Symfony\Component\Serializer\Serializer ;-)

e0ipso’s picture

wim leers’s picture

wim leers’s picture

The blackfire profile at #3014232-10: [regression] ResourceTypeRepository is significantly slower in JSON:API 2, becomes noticeable when handling hundreds of interlinked resources confirm this problem again, 9.74% of exclusive wall time is spent in \Symfony\Component\Serializer\Serializer::getNormalizer()!

That would IMHO make this worth working around in JSON:API. But hopefully #3014283: Use ResourceTypeFields to statically determine the field normalizer will solve this at a higher level, which would mean we wouldn't need to work around it :)

gabesullice’s picture

This is a core issue for the same concern ^

I do think there are meaningful and workable actions we can take in the JSON:API module to fix this w/o requiring upstream changes, but #3014283: Use ResourceTypeFields to statically determine the field normalizer is not yet one of them.

e0ipso’s picture

I disagree, #3014283: Use ResourceTypeFields to statically determine the field normalizer is both a meaningful and a workable action to solve this problem.

wim leers’s picture

#3017291: \Symfony\Component\Serializer\Serializer::getNormalizer() is slow was definitively shot down by both Drupal core and Symfony: Drupal 8 core cannot update to Symfony 4, and Symfony folks refuse to port the bugfix (they call it a "feature") to Symfony 3.

#3014232: [regression] ResourceTypeRepository is significantly slower in JSON:API 2, becomes noticeable when handling hundreds of interlinked resources was fixed yesterday.

AFAICT that leaves #3014283: Use ResourceTypeFields to statically determine the field normalizer as the next actionable thing. I think that #3014283-28: Use ResourceTypeFields to statically determine the field normalizer means that @gabesullice is +1 now, but I'm not entirely sure. But I think @gabesullice was perhaps pointing out that #3014283 needs #3014277: ResourceTypes should know about their fields to land first?

gabesullice’s picture

I think that #3014283-28 means that @gabesullice is +1 now, but I'm not entirely sure

+1 to what? I don't know what you're referring to.

I disagree, #3014283: Use ResourceFieldInterface to statically determine the field normalizer is both a meaningful and a workable action to solve this problem.

It's a meaningful and workable action, but it won't solve this problem. It will make it somewhat better.

Which is essentially what you said in #3014283-27: Use ResourceTypeFields to statically determine the field normalizer:

Wait. I agreed that we won't ever be able to solve the problem #3011497: [upstream] Too many calls to \Symfony\Component\Serializer\Serializer::getNormalizer() beyond a certain point. This issue will solve it for as much as we can do.

I think that there are more things that we can do. I have yet to collect and synthesize my thoughts about what I believe those things are though.

wim leers’s picture

+1 to what? I don't know what you're referring to.

In #9 on Nov 30 you linked to #3017291: \Symfony\Component\Serializer\Serializer::getNormalizer() is slow, suggesting that should happen as a solution for the cause rather than the symptom. In #3014283-28: Use ResourceTypeFields to statically determine the field normalizer you seemed to confirm @e0ipso in #3014283-27: Use ResourceTypeFields to statically determine the field normalizer that #3017291 cannot happen due to Drupal vs Symfony version and backporting constraints. And that #3014283 is therefore the only viable way forward out of that performance problem.

I think that there are more things that we can do.

Okay, so you think we can do even more. Great! 😀 But do you think that #3014283: Use ResourceTypeFields to statically determine the field normalizer is:

  1. at least a great interim step
  2. or, ideally, something that has to happen anyway?