Summary

Gander, the performance testing framework, has been part of Drupal since 10.2. We are using it to test performance in Drupal core and it already helped us identify and resolve multiple performance issues (#3410312: Flood database backend ::isAllowed() should call ::ensureTableExists(), #3410582: Optimize user logins by avoiding duplicate entity queries, #3349447: Improve Largest Contentful Paint in Umami front page, #3414287: Avoid reading session from the database multiple times during a big pipe request, ...).

We think that we should use it for Drupal CMS in order to make it the most performant open source CMS on the market and to make sure performance is not degraded over time.

Work to be done

  • Agree on a set of performance-focused goals for Drupal CMS. They can use Core Web Vitals or some other metric.
  • Provide monthly reports on the actual metrics vs. goals.
  • Ensure current CI environment is ready to run performance tests (most likely already the case).
  • Plan and write initial performance tests. Focus on base recipe initially and expand on optional recipes afterwards.
  • Set up a performance dashboard, similar to the one we have for Drupal core. In order to secure reliable results for the dashboard we would need a dedicated test runner (sponsorship/donation needed).
  • Support community with expanding and writing additional performance tests on an ongoing basis.
  • Support community with identifying and fixing performance issues on an ongoing basis.

Out of scope

Actively looking for an fixing performance issues is not in scope (providing support for the community to do so is).

Target milestone

TBD

Skills required

  • Experience planning, writing and maintaining performance tests.
  • General performance expertise.

Blockers / dependencies

TBD

Track owner

TBD (Tag1 is offering to sponsor work in this track.)

Proposal / roadmap

TBD

Comments

slashrsm created an issue. See original summary.

catch’s picture

catch
he/him
Primary language English
commented

Set up a performance dashboard, similar to the one we have for Drupal core. In order to secure reliable results for the dashboard we would need a dedicated test runner (sponsorship/donation needed).

The core performance dashboard currently requires ~4 test runs per hour to populate the graphs properly, this is due to some kind of prometheus/grafana quirk in graph rendering. A core test run takes about 8.5 minutes.

If we reduce the number of core test classes/methods, or if Drupal CMS performance tests can complete in ~5 minutes, we could run those tests on the same runner and still have results graphed - we just need to be able to run >= 8 tests per hour.

Alternatively it might be a one-line fix somewhere to make the grafana graphs work with less frequent test runs, the problem is figuring out which line ( and that it's not very easy to test this locally since it take a fair amount of time to generate data). If we can fix that, we can have multiple core branches and potentially Drupal CMS stable vs. HEAD and things like that.

thejimbirch’s picture

thejimbirch commented
Status: Needs review » Needs work

Marking as Needs Work based on the comments in #2

catch’s picture

catch
he/him
Primary language English
commented
Related issues: +#3496673: Require the performance tests to pass on CI

Next steps here:

#3496673: Require the performance tests to pass on CI

Also we should add performance tests for the optional recipes - either all at once or individually, need to talk to @phenaproxima about how to organise that.

catch’s picture

catch
he/him
Primary language English
commented
Related issues: +#3508161: Prepare for sending OpenTelemetry traces to the performance dashboard

I've refactored the core dashboard (https://gander.tag1.io/) to require less frequent test runs. It's currently relying on hourly tests of 11.1.x and 10.4.x instead of every 15 minutes, but theoretically we could run tests every four hours or twelve hours and it should still work now.

Next step for Drupal CMS is #3508161: Prepare for sending OpenTelemetry traces to the performance dashboard. Once that issue lands, we should be able to set up a pipeline schedule to populate the dashboard.

The remaining step after this would be adding some additional test coverage for optional recipes. I'm wondering if we can enable every optional recipe at once. InteractiveInstallTest installs all the recipes shown in the UI installer, but this isn't every optional recipe shipped with Drupal CMS. Not sure if there's a clean way to install 'everything in the recipes folder'