We have facets with a lot of values and have noticed a performance problem. Url::createFromRequest is called 1,200 times. That is due mostly to 600 calls to \Drupal\facets\Plugin\facets\url_processor\QueryString::buildUrls.

The offending line seems to be:

    $request = $this->request;
    if ($facet->getFacetSource()->getPath()) {
      $request = Request::create($facet->getFacetSource()->getPath());
      $request->attributes->set('_format', $this->request->get('_format'));
    }

Instead of recreating the request for each path, we could see if the new request object has been created before. This would act like a static cache of requests and add a performance boost.

Sample Blackfire.io output:

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

mglaman created an issue. See original summary.

mglaman’s picture

Issue summary: View changes
FileSize
49.55 KB
mglaman’s picture

Assigned: Unassigned » mglaman

The fix looks something like this:

    if ($facet_source_path) {
      if (isset($this->requestsByPath[$facet_source_path])) {
        $request = $this->requestsByPath[$facet_source_path];
      }
      else {
        $request = Request::create($facet_source_path);
        $request->attributes->set('_format', $this->request->get('_format'));
        $this->requestsByPath[$facet_source_path] = $request;
      }
    }

Git is currently down. I'll roll a patch later.

mglaman’s picture

Status: Active » Needs review
FileSize
1.36 KB

Here is a patch. I'll deploy it and run Blackfire.

borisson_’s picture

Status: Needs review » Reviewed & tested by the community

This looks very good, thank you! Setting this to rtbc - will probably commit this and tag a new release during drupalcamp belgium next week.

mglaman’s picture

Assigned: mglaman » Unassigned

Great! I've been getting a regular release for the project out. I'm still aiming to post an "after" profile.

mglaman’s picture

Status: Reviewed & tested by the community » Needs work

Sorry, I don't know what i'm thinking. ignore this comment.

mglaman’s picture

Status: Needs work » Reviewed & tested by the community
mglaman’s picture

Status: Reviewed & tested by the community » Needs review
FileSize
1.38 KB

Okay, I'm not crazy. The original patch did not work. The class is always instantiated as a new object, blowing the cache. We need a static variable in the method itself to preserve the cache.

mglaman’s picture

Status: Needs review » Needs work
FileSize
2.34 KB

Okay, there is more to it. It's also the large amount of calls to Url::createFromRequest.

This patch uses more caching, but it isn't elegant. I need to use this for now, but I'd like to find a better approach.

Maybe injecting cache.static? Or just a more elegant way of handling the checks.

The problem is this:

  • Constantly recreating request objects
  • Rebuilding the same URL via createFromRequest for each facet URL. Each facet URL needs two of these generated.
borisson_’s picture

This patch uses more caching, but it isn't elegant. I need to use this for now, but I'd like to find a better approach.

It doesn't bother me too much that is not elegant, but i guess it does need some inline documentation to explain what is going :)

mglaman’s picture

Assigned: Unassigned » mglaman

Okay, I'll clean it up so it makes more sense. We've had this in production for a week or so now. With 600 Facet objects the page went from 13s to 9s in Blackfire.

mglaman’s picture

Assigned: mglaman » Unassigned
Status: Needs work » Needs review
FileSize
4.14 KB

Okay, here is a cleaned up patch. Instead of stuffing static entries into the one method, I broke things out a bit. I felt like this made it easier to document and a less confusing. Otherwise, we would be defining static variables at the beginning of the method and using them sometime after, making it hard to comprehend -- especially since this pattern is not as common in Drupal as it is in PHP as a whole.

chr.fritsch’s picture

FileSize
4.39 KB

Rerolled

chr.fritsch’s picture

One question. Why is it called so often for you? For me QueryString::buildUrls is only called once per facet on every page load.

borisson_’s picture

Status: Needs review » Needs work
Issue tags: +Needs reroll

Patch doesn't apply anymore as I committed other stuff first. Sorry about that.

michaellenahan’s picture

FileSize
4.68 KB

Rerolled

michaellenahan’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 17: 3013991-17.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

borisson_’s picture

The patch in #17 seems to break some of our testcoverage. This could possibly be because our testcoverage isn't that great. But it could also be a legit issue. I don't really have the time to investigate that now.

legolasbo’s picture

Status: Needs work » Needs review
FileSize
5.46 KB

The patch was using regular static variables, which caused interference when running tests because static variables are not reset within tests. I used drupal_static instead to ensure that the tests never run into a cached instance.

borisson_’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: -Needs reroll

Awesome - this should still be a lot faster.

  • borisson_ committed 799ea27 on 8.x-1.x authored by legolasbo
    Issue #3013991 by mglaman, chr.fritsch, michaellenahan, legolasbo,...
borisson_’s picture

Status: Reviewed & tested by the community » Fixed

Committed and pushed, thanks everyone.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.