Problem/Motivation

There are multitude of normalizers in the system I've seem 33 in my code base registered in services.yml somewhere. The symphony component finds the first one that can do the job and then uses it. There is no caching involved, so any normalization processing will involve this loop over the defined normalizers.

In my case this is causing ~20% of the total request/response time :(.

Proposed resolution

Initial thoughts:
Can we just decorate the base service or put the jsonapi one before it, so we can spare the 250ms meaningless calls to Drupal\serialization\Normalizer\NormalizerBase::supportsNormalization

If we know we have only this 2 normalizers and they do all the work needed, can we somehow spare the repeated calls in there. and use only the more powerful one - jsonapi and force TRUE all the time, essentially zeroing out the supportsNormalization overhead.


After all has settled and all is done, the root cause is found in #3017291: \Symfony\Component\Serializer\Serializer::getNormalizer() is slow.
As that issue is currently blocked there is a patch for symphony 3.4 serializer in comment #8.

Remaining tasks

Discussion, patch direction, patch...

User interface changes

TBD.

API changes

TBD.

Data model changes

TBD.

Comments

ndobromirov created an issue. See original summary.

wim leers’s picture

Title: Fetching normalizers seems slow » [upstream] Fetching normalizers seems slow
Related issues: +#3014283: Use ResourceTypeFields to statically determine the field normalizer

First of all: thank you so much for digging in deeper in performance issues! We're finally ready to be doing that, and as I wrote in https://wimleers.com/blog/state-of-jsonapi-2018-11: We’re sure there’s more low-hanging fruit — looks like you're finding it 😃 🎉

This is "just" a consequence of using the Symfony serialization system. JSON:API isn't doing anything special. It's the upstream code that is of questionable quality, sadly, and the version that Drupal core is on is now minimally maintained.

So I'm hesitant to add more optimizations to work around upstream flaws rather than solving it at the root, by stopping to use those upstream APIs.

Therefore I have this question for you: Why do you feel this should be a separate issue instead of being added to #3014283: Use ResourceTypeFields to statically determine the field normalizer?

ndobromirov’s picture

I've already linked it there. Based on a gut feeling this seems the same - we aim at reducing normalizer look-ups.

ndobromirov’s picture

Yeah but Drupal has something like 30+ normalizers registered in the container. So getNormalizer loops 14k times * 30 => 300-400k supports normalization calls out of the blue... Symphony one is not a bad system, we just need to put the most important one first, so we spare the excessive looping.

ndobromirov’s picture

Status: Active » Needs review

As we essentially out-scoped this form the other issue and opened a core to resolve this directly, should this be closed?

e0ipso’s picture

Status: Needs review » Closed (won't fix)

Let's close. We have ample options to track this.

ndobromirov’s picture

StatusFileSize
new149.14 KB
new148.75 KB

As it was found in the other issue(s) there is a performance fix upstream in symphony (4.1).

This is the tuned version of the hack from the other issue that resolved a BIG portion of the overhead from Symfony/Component/Serializer/Serializer::getNormalizer(...) class.

# ...
    private $normalizersCache = [];

    /**
     * Returns a matching normalizer.
     *
     * @param mixed  $data    Data to get the serializer for
     * @param string $format  Format name, present to give the option to normalizers to act differently based on formats
     * @param array  $context Options available to the normalizer
     *
     * @return NormalizerInterface|null
     */
    private function getNormalizer($data, $format, array $context)
    {
        $type = \is_object($data) ? \get_class($data) : 'native-' . \gettype($data);
        $cacheId = "normalizer:$type:$format";
        if (isset($this->normalizersCache[$cacheId]) || array_key_exists($cacheId, $this->normalizersCache)) {
          return $this->normalizersCache[$cacheId];
        }

        foreach ($this->normalizers as $normalizer) {
            if ($normalizer instanceof NormalizerInterface && $normalizer->supportsNormalization($data, $format, $context)) {
                return $this->normalizersCache[$cacheId] = $normalizer;
            }
        }

        return $this->normalizersCache[$cacheId] = null;
    }
# ...

Responses from jsonapi are exactly the same.

Here is before...

and after...

... the code above was applied.

I plan to have a patch posted here for that library.
Even if not recommended, this is speeding-up the ::getNormalizer() method with close to a second of run-time for my case or about 30+ times.

ndobromirov’s picture

Here is the promised patch...
I am not putting into needs review as it's for my own use (unless anyone likes it) :D


Hopefully symphony will be updated in D8 soon so this will not be needed at that time...
renrhaf’s picture

Thanks ndobromirov ! With your patch I load 50% faster on normalizer heavy pages !
But be careful applying this code, because specific normalizers targeting data with supportsNormalization method could not be cached.

ndobromirov’s picture

@Renrhaf thanks for the feedback.
You should consider the latest version of jsonapi or paching the one in 8.7 with #2819335: Resource (entity) normalization should use partial caching.

Adding the core issue that caused this, as a related one.
Updated issue summary with note about the PoC patch in #8.

tamnv’s picture

Hi everyone,

I created new patch for everyone who's looking for patch for drupal 8.6.17.

ndobromirov’s picture

And another that will work for symfony/serializer:3.4.37.