Problem/Motivation

Follow-up to #3388365: Distribute @group #slow tests between test runners and mark more tests, there are a handful of stragglers.

Steps to reproduce

Proposed resolution

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

Issue fork drupal-3389839

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

catch created an issue. See original summary.

catch’s picture

Once the worst culprits in #3389281: [meta] Refactor ultra-slow tests are done, any remaining slower tests tend to stick out towards the end of test runs. We probably want to mark most functional tests with more than 5 methods as #slow. The limit is that we don't wan to mark more tests slow than concurrency * parallel job runs, because that could push one of the very slow tests into the second batch of tests, which would then make the total run slower again.

So if we have 6 parallel functional test runs at concurrency 16 each, the absolute maximum number of @group #slow classes should be 6 * 16 = 96. Hopefully we'll end up with a lot lower than that of course.

catch’s picture

Status: Active » Needs review

Added a few here.

smustgrave’s picture

Status: Needs review » Reviewed & tested by the community

Seems a lot of tests are slow haha. looks good though.

catch’s picture

Status: Reviewed & tested by the community » Needs review

Had some commits locally to push, back to needs review but just more of the same.

https://git.drupalcode.org/project/drupal/-/pipelines/24325 shows the results (combined with several other issues) - 11m1s for a full test run.

If you look at the individual jobs on https://git.drupalcode.org/project/drupal/-/pipelines/24325 and ctrl-F for 'test run duration' you can see all the functional tests are finishing in 5-6 minutes. It is not perfectly balanced and there are still one or two tests that are 'ultra slow' and overshoot all the others, but it's a lot closer than without the MR when they could vary more like 4-8 minutes.

Kernel tests and Functional JavaScript tests are also 'balanced enough'. We can probably do some more work on kernel tests to mark the slow ones there, but could happen in a follow-up.

smustgrave’s picture

Status: Needs review » Reviewed & tested by the community

Kept this open for close to 24 hours in case you found more. But going to remark it now.

catch’s picture

Yes I am properly done for now :)

lauriii made their first commit to this issue’s fork.

lauriii’s picture

Status: Reviewed & tested by the community » Fixed

Looks like we have 92 functional tests marked as #slow after this so the number of slow tests remain barely under the threshold mentioned in #3.

Committed 4316924 and pushed to 11.x. Thanks!

  • lauriii committed 43169249 on 11.x
    Issue #3389839 by catch: Mark some more tests #slow
    
catch’s picture

fwiw with functional test concurrency, the current numbers look like this:

HEAD:

 6*32=192

After #3388952: Tweak gitlab CI concurrency, parallelism and test running order for 11 minute test runs it's lower but still high.

 to 7 * 24=168

So still a lot of headroom to add #slow unless we significantly constrain the number of tests we run at once. Obviously it would be better if we made the actual tests less slow too so they don't need the tag.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.