Problem/Motivation

The 3.0.x branch has no GitLab CI configuration. It is the branch that supports Drupal 10 and 11, so it is the branch nearly every site running this module is on — and nothing on it is verified automatically. Merge requests get no pipeline, and maintainer commits get no pipeline. Everything that has landed recently was verified only by whatever the contributor happened to run on their own machine.

By contrast, 4.0.x (the Drupal 12 branch) picked up a working .gitlab-ci.yml during the Drupal 12 compatibility work and does run pipelines on every push and merge request.

Steps to reproduce

1. No CI configuration on 3.0.x. The root of 3.0.x carried only the retired DrupalCI configuration (drupalci.yml plus a stray drupalci.yml_.txt), while 4.0.x has .gitlab-ci.yml. DrupalCI was decommissioned in 2023, so those files do nothing.

2. No pipeline had ever run for 3.0.x. Both /merge_requests/14/pipelines and /pipelines?ref=3.0.x returned an empty array. All 24 pipelines in the project belonged to 4.0.x refs.

3. Even with CI, 3.0.x would have executed zero tests. Its two test files were named with a plural Tests suffix, which PHPUnit's *Test.php discovery never collects. Worse, the Drupal Association template gates the phpunit job on exists: '**/tests/**/*Test.php', so the job would not have been created at all and the pipeline would have gone green with one fewer job. MR !14 fixed the naming.

4. Recent fixes landed with no automated verification. Commit 039cb78 (issue 3531242) and MR !14 (issue 3614068) both went in without a pipeline.

Proposed resolution

Add a .gitlab-ci.yml to 3.0.x modelled on 4.0.x's, but without the Drupal-12-specific overrides (the job-level DRUPAL_CORE: $CORE_MAJOR_DEVELOPMENT, _TARGET_PHP: $CORE_NEXT_PHP_MIN, IGNORE_PROJECT_DRUPAL_CORE_VERSION and MINIMUM_STABILITY_OVERRIDE). This branch declares core_version_requirement: ^10 || ^11, so it should take the template defaults for both core and PHP. Also delete drupalci.yml and drupalci.yml_.txt.

Two variables are not what you would expect, and both were discovered the hard way while implementing this. See comments below for the full diagnosis; the short version:

  • _PHPUNIT_TESTGROUPS must be '--all', not 'layout_builder_kit'. PHPUnit 10 and later ignore doc-comment metadata on any class that carries at least one PHP attribute. The Kernel classes carry #[RunTestsInSeparateProcesses], so their @group annotation was invisible under Drupal 11 and a --group run collected 2 tests instead of 12. Drupal 10's PHPUnit 9 has no attribute support at all, so it still read the annotation and collected all 12 — the two majors disagreeing is what exposed the bug. '--all' is the template's value meaning "add no --group filter"; the invocation is already scoped to the module directory.
  • OPT_IN_TEST_PREVIOUS_MAJOR: '1' is not sufficient on its own. _AUTORUN_PREVIOUS_MAJOR defaults to none, which leaves the Drupal 10 jobs as manual stubs that never run — indistinguishable from real D10 coverage unless you look. Both variables are required.

The working configuration:

include:
  - project: $_GITLAB_TEMPLATES_REPO
    ref: $_GITLAB_TEMPLATES_REF
    file:
      - '/includes/include.drupalci.main.yml'
      - '/includes/include.drupalci.variables.yml'
      - '/includes/include.drupalci.workflows.yml'

variables:
  SKIP_NIGHTWATCH: '1'
  _PHPUNIT_TESTGROUPS: '--all'
  OPT_IN_TEST_PREVIOUS_MAJOR: '1'
  _AUTORUN_PREVIOUS_MAJOR: 'all'

Remaining tasks

  • Review and merge MR !16.
  • The advisory lint jobs (phpcs, phpstan, stylelint, eslint) fail on pre-existing debt. They are allow_failure: true so nothing blocks. Cleaning them up and only then making each one gate is tracked separately in #3614242.
  • Once this lands, rebase #3614245's MR !17 so the Book Navigation fix gets a real pipeline on both majors rather than local verification only.

Scope note

MR !16 grew beyond adding CI, deliberately. Turning CI on is only worth something if it runs the tests, and two problems had to be fixed before it did:

  • LayoutBuilderKitUITest assigned $this->adminUser without declaring the property. On PHP 8.2+ that is a deprecation, Drupal's PHPUnit configuration treats printed output as risky, and the job exits non-zero. Drupal 10 runs PHP 8.1 and never saw it, so this was Drupal 11 only.
  • #[Group('layout_builder_kit')] was added to the Kernel classes so a --group run also works, alongside the retained @group docblocks for PHPUnit 9.

It also backports the unit and kernel tests that only existed on 4.0.x, taking the branch from 12 collected tests to 28, and ports a trimmed .cspell.json so that job produces a real signal rather than permanent advisory noise.

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

aangel created an issue. See original summary.

  • aangel committed 7f86d5a7 on 3614091-gitlab-ci
    Issue #3614091: Collect all module tests under PHPUnit 11 and declare $...

  • aangel committed 64304d4e on 3614091-gitlab-ci
    Issue #3614091: Hide Book Navigation without Book, and backport kernel...

  • aangel committed 13fb65be on 3614091-gitlab-ci
    Issue #3614091: Backport the unit tests from 4.0.x to 3.0.x.
    
    3.0.x had...

  • aangel committed 3a0ef69a on 3614091-gitlab-ci
    Issue #3614091: Backport the remaining kernel tests from 4.0.x....
aangel’s picture

Issue summary: View changes
aangel’s picture

Issue summary: View changes
aangel’s picture

Implemented in MR !16. Recording the diagnosis, because two of the variables in the original proposal were wrong in ways that produce a pipeline which looks like it works and does not.

Attempt 0 — a branch push produces no pipeline at all. The templates' workflow rules only fire on merge requests, tags, schedules, web triggers and the default or release branches. A feature branch matches none of them. You cannot validate a CI change by pushing a branch; opening the MR is what triggers it.

Attempt 1 — _PHPUNIT_TESTGROUPS: 'layout_builder_kit', copied from 4.0.x, silently collected 2 tests out of 12.

phpunit                    (Drupal 11)  Tests: 2, Assertions: 7 ... Risky: 1   FAILED
phpunit (previous major)   (Drupal 10)  OK (12 tests, 39 assertions)           passed

Root cause: PHPUnit 10 and later ignore doc-comment metadata on any class that carries at least one PHP attribute. Both Kernel classes carry #[RunTestsInSeparateProcesses], so under Drupal 11 their @group layout_builder_kit annotation was invisible and the --group filter collected only the two Functional tests. Drupal 10 ships PHPUnit 9, which has no attribute support at all, so it still read the annotation and collected all twelve.

The two majors disagreeing is the only reason this was caught. Had this branch tested a single major, it would have shipped a 2-test suite reporting success. The fix is _PHPUNIT_TESTGROUPS: '--all' — the template's value meaning "add no --group filter" — since the invocation is already scoped to the module directory. The Kernel classes also gained #[Group('layout_builder_kit')] alongside their existing docblocks, so a group run works too.

Attempt 1 also exposed a second, Drupal-11-only failure. LayoutBuilderKitUITest assigned $this->adminUser without declaring the property. On PHP 8.2+ that is a deprecation; Drupal's PHPUnit configuration treats printed output as risky, and the job exits non-zero — so OK, but there were issues! still meant EXIT_CODE=1. Drupal 10 runs PHP 8.1 in CI and never saw it.

OPT_IN_TEST_PREVIOUS_MAJOR: '1' is not sufficient on its own. _AUTORUN_PREVIOUS_MAJOR defaults to none, which leaves the Drupal 10 jobs as manual stubs that never run. The pipeline then shows D10 job names and green overall while providing no D10 coverage whatsoever — indistinguishable from the real thing unless you open a job. Both variables are required.

Final result, pipeline on the merge ref, verified from the job traces rather than the aggregate status:

composer                   Locking drupal/core (11.4.4)
composer (previous major)  Locking drupal/core (10.6.13)
phpunit                    Tests: 28, Assertions: 89        (PHPUnit 11.5.56, PHP 8.3.33)
phpunit (previous major)   OK (28 tests, 91 assertions)     (PHPUnit 9.6.35)

The JUnit artifacts from both jobs were diffed: identical test-case sets, zero symmetric difference. The assertion count differs by two purely from PHPUnit 9 versus 11 internal counting, not a coverage gap.

The advisory lint jobs (phpcs, phpstan on both majors, stylelint, eslint) fail on pre-existing debt and are allow_failure: true, so nothing blocks. That cleanup is #3614242. cspell passes.

The general lesson, since it cost three attempts here and has a sibling on 4.0.x: a CI configuration that reads correctly has been wrong more often than not on this project, and only the job log settles it. On 4.0.x the same shape of bug meant a Drupal-12-only branch was silently built against Drupal 11 for its entire history. Check the executed test count and the resolved dependency versions in the trace, not the badge.

  • aangel committed 7d5410bd on 3.0.x
    Issue #3614091: Add GitLab CI to 3.0.x and make the test suite actually...
aangel’s picture

Status: Active » Fixed

Merged.

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.