Problem/Motivation

When we moved lint jobs to the parent, unit tests became a blocking step between linting and the rest of the tests. This means a pipeline looks like the following:

composer + yarn -> lint -> composer + yarn -> unit tests -> other tests.

We use several runners for the 'lint' step and several runners for the 'other tests' step, but only two runners for composer + yarn and only one runner for phpunit.

According to @nnewton there is a 20 minute lease on runners, so if there aren't other jobs outside a pipeline for them to pick up, they'll be sitting idle for about 2-4 minutes while phpunit runs.

Since unit test failures isn't the common case (i.e. they usually pass), we should remove -with-unit-tests from the other tests, and just make everything depend on lint (which it defacto does due to the parent/child split). phpcs and phpstan increasingly catch the sort of silly mistakes that would also cause phpunit test failures.

This way there's a shorter gap between linting and other steps, meaning the runners don't sit there doing nothing inbetween.

If we want to block tests on unit tests again, we should consider duplicating them into the main job too, so that they run alongside linting and block there, and then again with the PHP version, however the amount of boilerplate YAML to get run-tests to work in that step is significant.

This should result in 2-4 minute quicker overall test runs.

Steps to reproduce

Proposed resolution

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

CommentFileSizeAuthor
#6 Screenshot 2023-09-18 at 19.39.15.png33.17 KBfjgarlin

Issue fork drupal-3387706

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

catch created an issue. See original summary.

catch’s picture

Title: Run unit tests during lint step as well as child jobs » Don't make other tests depend on PHPUnit
Issue summary: View changes
Related issues: +#3386217: \Drupal\Tests\Core\Test\PhpUnitCliTest requires a database and webserver - move to a functional test
catch’s picture

Status: Active » Needs review
Parent issue: » #3386474: [omnibus] Speed up gitlab ci runs
fjgarlin’s picture

Status: Needs review » Needs work

Comments in MR.

fjgarlin’s picture

Status: Needs work » Reviewed & tested by the community
StatusFileSize
new33.17 KB

Changes look good, all feedback was addressed and pipelines running as expected: https://git.drupalcode.org/project/drupal/-/pipelines/21204

All tests running at the same time, so no waiting for phpunit:
tests

RTBC.

larowlan’s picture

Version: 11.x-dev » 10.1.x-dev
Status: Reviewed & tested by the community » Fixed

Thanks, committed to 11.x and backported to 10.1.x

  • larowlan committed 4f6fe130 on 10.1.x
    Issue #3387706 by catch, fjgarlin: Don't make other tests depend on...

  • larowlan committed 55925d1c on 11.x
    Issue #3387706 by catch, fjgarlin: Don't make other tests depend on...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.