Problem/Motivation
run-tests.sh splits test classes between parallel CI jobs with WorkAllocator. It deals out #slow classes round robin, then the other classes the same way. Round robin ignores how much work each job already has. Each round gives the largest class to the first job, and both the slow and normal passes start at the first job, so the first jobs get more work.
Steps to reproduce
Proposed resolution
Each test class now gets a weight: its test case count, multiplied by 8 for #slow classes. Classes are then sorted by weight, heaviest first. Each class goes to the job with the lowest total weight so far. Ties are broken by class name so sorts are consistent.
With the help of Claude Code, a simulation took test run times from three daily main pipelines (996748, 997801 and 999018). Mean time of the slowest job, before and after:
- Kernel: 146 s to 143 s
- Functional: 236 s to 203 s
- FunctionalJavascript: 252 s to 238 s
EntityDefinitionUpdateMultipleTypesTest takes 128 seconds which sets the floor for kernel tests.
Remaining tasks
User interface changes
Introduced terminology
API changes
Data model changes
Release notes snippet
Issue fork drupal-3630648
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #3
longwaveComment #4
longwaveI think we can also safely reduce the number of kernel jobs from 4 to 3, as functional test jobs all take significantly longer.
Comment #5
longwavehttps://git.drupalcode.org/project/drupal/-/pipelines/1000957 completed in 5m19
Kernel tests took 143s, 118s, 128s
Functional tests took 123-255s
FunctionalJavaScript tests took 215s, 253s, 215s
We could even try reducing kernel to 2, but I think in some cases that will make it run slightly longer.
Comment #6
mondrakeAdding some related issue in this 'test work allocation optimization' space that have similar intent, ftr and later ease of reference.
I think this is good, probably we just need to try it and see its performance. The test coverage is also good to ensure isolation between the bins.
The only thing I would suggest here is to add a 'weight' column to the debug-discovery report, near to the 'count' column, so to make more visually clear the logic for the sorting, as this is no longer obvious.
Comment #7
mondrakeComment #8
longwaveAdded a weight column and simplified the code a bit.
We can try tweaking the magic number 8 but I'm not sure exactly how much difference it will make.
Comment #9
mondrakeTweaking the weight of the #slow tests would probably make sense if we end up with major WT differences between bins of the same test type.
Let’s see first; time will tell. RTBC