Problem/Motivation

After #3614401: Allow install profiles to opt out of being installed and #3614153: Install system module alongside other modules in the installer it looks like we can reduce the CPU and concurrency for functional tests without an increase in wall times.

At the moment, core pipelines make the following simultaneous CPU requests (e.g. ignoring lint jobs which run before):

Functional js: 36
Functional: 128
Kernel: 48
Build: 8
Nightwatch: 16

This is a total CPU request of 236.

The maximum CPU for an AWS instance type is 192 according to https://aws.amazon.com/ec2/instance-types/general-purpose/

This issue reduces the CPU request by 47.

236 - 47 = 189

Steps to reproduce

Proposed resolution

Nightwatch tests now regularly finish in 2m-2m30s so move them to the penultimate job next to build tests which finish in a similar amount of time, and reduce the CPU request since it's not making a big difference.

Mark a few functional tests with group #slow so that the jobs are more evenly balanced.

Bring the CPU request for functional tests down from 14 to 10.

Overall reduction of 47 CPU request for every core test pipeline.

Remaining tasks

At least some of these issues should land first so we get a realistic idea of what we need:

#3615455: Install database driver modules alongside other modules
#3614401: Allow install profiles to opt out of being installed
#3614825: Allow additional modules to be installed alongside the profile modules
#3615423: Avoid rebuilding the container in tests just to create a directory

User interface changes

Introduced terminology

API changes

Data model changes

Release notes snippet

Issue fork drupal-3614817

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

catch created an issue. See original summary.

catch’s picture

catch
he/him
Primary language English
commented
Title: [PP-2] Reduce parallel function test jobs » [PP-2] Reduce parallel functional test jobs
catch’s picture

catch
he/him
Primary language English
commented

Opened #3614825: Allow additional modules to be installed alongside the profile modules too. A combined pipeline of the three issues + some individual test optimisations has functional tests finishing in about 90s each with no changes to parallel. Going to try going down to 4 from 8 and see how it looks.

catch’s picture

catch
he/him
Primary language English
commented
Title: [PP-2] Reduce parallel functional test jobs » [PP-3] Reduce parallel functional test jobs
Status: Active » Postponed

Went for reducing the CPU and concurrency rather than the number of jobs - this is on the basis that smaller jobs are more likely to fill space on machines that are already in use - e.g. if a machine has 10 CPUs spare it can fit a 10 CPU request job but not a 16 CPU request job.

This should give us a baseline of the jobs taking a while, but then rebasing when the three issues in front of this land should get us back to relatively speedy.

catch opened merge request !16576

catch’s picture

catch
he/him
Primary language English
commented
Title: [PP-3] Reduce parallel functional test jobs » [PP-3] Reduce functional test job CPU and concurrency
catch’s picture

catch
he/him
Primary language English
commented
Related issues: +#3574200: Get rid of 2-phase kernel boot in installer

#3574200: Get rid of 2-phase kernel boot in installer looks like it should allow a fourth container rebuild to be dropped.

catch’s picture

catch
he/him
Primary language English
commented
Related issues: +#3615979: [meta] Reduce container rebuilds in the installer and functional/kernel tests
catch’s picture

catch
he/him
Primary language English
commented
Issue summary: View changes
catch’s picture

catch
he/him
Primary language English
commented
Issue summary: View changes
catch’s picture

catch
he/him
Primary language English
commented
Title: [PP-3] Reduce functional test job CPU and concurrency » Reduce functional test job CPU and concurrency
Status: Postponed » Needs review

OK I think at least a first round of this is unblocked at this point.

Reduced CPU and concurrency for functional tests a fair bit.

We've dropped some nightwatch tests due to removed modules with more to go, so moved those to the end because they finish faster now.

Kept functional js tests where they are because those still finish last at the moment.

Once some more issues land we might be able to do even more, but might as well see how this goes for a while.

dcam’s picture

dcam commented
Status: Needs review » Reviewed & tested by the community

Based on the pipelines that have run for this issue and other recent pipelines for main it appears that this is achieving the goal of reducing concurrency without impacting the wall time. So I think this is OK. I didn't find any issues with the moved configuration.

catch’s picture

catch
he/him
Primary language English
commented
Issue summary: View changes

Rebased again now that #3615455: Install database driver modules alongside other modules is in and updated the issue summary.

catch’s picture

catch
he/him
Primary language English
commented
Issue summary: View changes
catch’s picture

catch
he/him
Primary language English
commented

Noticed on https://git.drupalcode.org/issue/drupal-3589236/-/jobs/11790005 that we currently have five slow build tests, so went ahead and increased concurrency to 6 for those from four, but without raising the CPU because we already have 8 allocated there.

catch’s picture

catch
he/him
Primary language English
commented

Last commit worked for build tests - tests themselves finished in 1m44s https://git.drupalcode.org/project/drupal/-/jobs/11875750 Leaving RTBC because this is a one character change.

longwave’s picture

longwave
he/him
Primary language English
Location UK
commented
Status: Reviewed & tested by the community » Fixed

Committed and pushed 29d9d03650f to main. Thanks!

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.

  • longwave committed 29d9d036 on main 
    ci: #3614817 Reduce functional test job CPU and concurrency

By: catch

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.