Problem/Motivation

Currently during a route rebuild the module loops through each view, then gets the entity type and bundles for the view, and the grabs the resource names for each entity/bundle combination:

        $resource_types = array_map(function ($bundle) use ($entity_type) {
          return $this->resourceTypeRepository->get($entity_type, $bundle)->getTypeName();
        }, $bundles);

During normal usage this works fine, but during database updates it causes a massive slowdown. If a module does any sort of update that requires a route rebuild, the module can take over 45 minutes to complete on a moderately sized site (46 views with `13 content types in my case). This is because during database updates the cache backend is switched to UpdateBackend, which prevents cache writes. This makes it so that each call to the resourcetype repository involves it rebuilding its cache, which slows down things considerably.

Proposed resolution

The easiest fix for now would be to hold onto the $resources_types returned for each entity type, and use them for any other views that share that entity type. This way if you have 10 views that are node based, for example, you're only paying once for the performance hit to get the resources names. During normal use it doesn't make a huge difference but during database updates you get a massive performance increase. That's what the attached patch does.

CommentFileSizeAuthor
jsonapi_views-speed-up-route-rebuild.patch1.5 KBjacobbell84

Issue fork jsonapi_views-3484714

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

jacobbell84 created an issue. See original summary.

jlenni’s picture

jlenni commented

Thank you, Jake, for the patch! This is already a life saver

Identifying the issue was quite challenging - unexpectedly, updb during deployment took 26 minutes, leaving me completely in the dark.

Would it be beneficial if we could limit the processing to views where "Exposed via JSON" is enabled?
Currently, as I see it, all active views are being processed, regardless of whether "Exposed via JSON" is enabled or disabled.

chodec’s picture

chodec commented

Omg, I have spent several hours to trace the source of this problem.
When I realized, that cause of all is JSON:API Views module, I found this patch. :) Thank you Jake!

yahyaalhamad’s picture

yahyaalhamad
Location Palestine
commented
Status: Needs review » Reviewed & tested by the community

Thank you for the patch, I had the same experience as #3

yahyaalhamad’s picture

yahyaalhamad
Location Palestine
commented
Priority: Normal » Major
daniel.pernold’s picture

daniel.pernold commented

Had the same problem debugging to hell. Found this issue too late, but added a MR here: https://www.drupal.org/project/jsonapi_views/issues/3580102

anybody’s picture

anybody
Primary language German
Location Porta Westfalica
commented

@daniel.pernold could you maybe instead turn the patch into a MR here to speed up fixing that? The other issue should be closed as duplicate then I think.

Could someone perhaps ask the maintainers if this module is still actively maintained or if they could have a look?

deciphered made their first commit to this issue’s fork.

deciphered opened merge request !25

deciphered’s picture

deciphered
Primary language English
Location Ballarat, Victoria, Australia
commented

Thanks jacobbell84 for the patch, and jlenni, chodec, yahyaalhamad, and daniel.pernold for confirming the impact.

Applied the patch as-is and added test coverage for the cache-hit path, since the module's existing fixture never had more than one view. Measured it at the scale from the issue: ResourceTypeRepository::get() called 637 times before the fix, 13 times after.

Tests green on D10 and D11.

  • deciphered committed 6245b5e5 on 8.x-1.x 
    test(#3484714): cover route building when views share an entity type

  • deciphered committed 6e1f42e6 on 8.x-1.x 
    fix(#3484714): cache resource type names per entity type
deciphered’s picture

deciphered
Primary language English
Location Ballarat, Victoria, Australia
commented
Status: Reviewed & tested by the community » Fixed

Commit and fixed, will be part of the upcoming 8.x-1.2 release.

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.