Problem/Motivation

The scheduled Performance tests job has failed every time it has run since 31 July 2026. In the last 24 scheduled pipelines it ran 20 times and failed 20 times. The three green pipelines are the 01:03 schedule, where the job stays manual and never runs.

The last pass was job 11270826, in pipeline 907399. That pipeline and 907630 (fail) both ran commit a0efbf39ff, with no commit between them, so nothing in core changed.

Two failures alternate on the same commit:

  • WebDriver\Exception\UnknownError: tab crashed
  • StylesheetBytes between 17506 and 35500, against 40800 ±2000

Chrome keeps its shared memory in /dev/shm, which is 64 MB in the CI services. MINK_DRIVER_ARGS_WEBDRIVER on line 87 does not pass --disable-dev-shm-usage, so the Umami test runs out of it. When Chrome drops responses, collectNetworkData() sees fewer stylesheet URLs and the byte count falls, which is the second failure.

Steps to reproduce

Reproduced on selenium/standalone-chrome:133.0 with /dev/shm at 64 MB, three runs per cell:

test session idle 8 concurrent sessions
flag set pass pass
flag unset fail fail

Every local failure raised the same exception as CI.

Proposed resolution

Add --disable-dev-shm-usage to line 87. The Nightwatch job already passes it on line 302 of the same file, where it writes DRUPAL_TEST_WEBDRIVER_CHROME_ARGS. Thresholds stay unchanged.

Issue fork drupal-3615256

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

sjpagan created an issue. See original summary.

sjpagan’s picture

sjpagan commented
Issue summary: View changes

sjpagan opened merge request !16607

sjpagan’s picture

sjpagan commented
Issue summary: View changes

The job needs the performance-test-runner tag, so it cannot run from an issue fork. I played it on the merge request, it sat queued for 1256 s with no runner, and I cancelled it.

Reproduced locally instead, on selenium/standalone-chrome:133.0, three runs each:

flag /dev/shm result
set 64 MB 3/3 pass
unset 64 MB 3/3 fail, ~34 s in
unset 2 GB 3/3 pass

The third row isolates the cause: without the flag, but with more shared memory, the test passes. So the failure follows the size of /dev/shm. Two of the three failures raise tab crashed.

This is a local reproduction, not a green CI run.

sjpagan’s picture

sjpagan commented
Status: Active » Needs review
smustgrave’s picture

smustgrave commented

Think this is a won't fix, if main was broken think we would know as all MRs would be failing.

catch’s picture

catch
he/him
Primary language English
commented

@smustgrave there is a scheduled performance test pipeline which runs only performance tests, and populates https://gander.tag1.io
See https://git.drupalcode.org/project/drupal/-/pipeline_schedules for the full list of scheduled pipelines

This one has been failing for a couple of weeks. Given this is a one liner and we can't verify it on the MR pipeline (by design, since that stops the dashboard being populated by MRs by mistake), I'm going to go ahead and commit this from needs review directly.

  • catch committed b45914ff on main 
    fix: #3615256 Performance tests job fails on main: Chrome runs without...
catch’s picture

catch
he/him
Primary language English
commented
Status: Needs review » Fixed

Went ahead and committed/pushed to main, thanks!

Manually triggered a performance test pipeline and it's all green again https://git.drupalcode.org/project/drupal/-/jobs/11592191

I think we just missed disabling shm when setting it up, but it worked without it so didn't notice until now.

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.

smustgrave’s picture

smustgrave commented

Thanks for explaining that catch was not aware.