The 2026 Drupal Business Survey is open. Your response will help shape the future of the Drupal business ecosystem.Problem/Motivation
run-tests.sh hardcodes using core's phpunit.xml.dist file when invoking PHPUnit subprocesses.
It would be good to allow passing in alternative (e.g. module specific) xml configuration files to allow tailoring test execution behaviour, as this seems to be becoming more and more a need as PHPUnit evolves.
Proposed resolution
- Add an additional
--phpunit-configurationargument for run-tests.sh, adjust GitLabCI scripts to add a variable holding PHPUnit configuration file path and using it when executing run-tests.sh. - Copy current
phpunit.xml.distconfiguration file to a file named.phpunit-10.xmlso that tests that need to run with PHPUnit 10 can reference that file - Edit
phpunit.xml.distconfiguration file to remove duplicate directories that are generating test discovery warnings under PHPUnit 11 - Dropped the earlier
.phpunit-next.xmlfile and made performance test point to the main core PHPUnit configuration now that it's PHPUnit 11 fully compliant.
Remaining tasks
User interface changes
Introduced terminology
API changes
Data model changes
Release notes snippet
| Comment | File | Size | Author |
|---|
Issue fork drupal-3530113
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
- 3530113-allow-indicating-alternative
changes, plain diff MR !12388
Comments
Comment #3
mondrakeComment #4
mondrakeComment #5
smustgrave commentedSeems pretty straight forward, didn't seem to break gitlab :). Not sure if this kind of change needs a CR but addition of --phpunit-configuration seems fine to me.
Comment #6
jonathan1055 commented@mondrake and @smustgrave, Would you like me to use this patch in a Contrib pipeline to try it and see if I can use an alternative phpunit.xml when runing phpunif with run-tests.sh?
Comment #7
mondrake#6 would be great!! Thanks in advance
Comment #8
smustgrave commentedYea that would be cool to see!
Comment #9
jonathan1055 commentedI have the mechanics of this working now in Scheduler MR194. The
patchhas to be allowed to fail because the two.gitlabfiles are not present, but that does not matter, the other two files are patched OK. I have added a custom phpunit config, calledcustom.xmljust to make sure we can tell it is distinct. I have defined a testsuite in it, but using--testsuitegivesERROR: Unknown argument '--testsuite'as that is not a recognised parameter inrun-tests.shhttps://git.drupalcode.org/project/scheduler/-/jobs/5637581
I then tried just using the name of the testsuite, which is 'only-token-test' and that gives
ERROR: Test group not found: only-token-testbecause of course the default type of filter is on test group, not test suite.https://git.drupalcode.org/project/scheduler/-/jobs/5637673
Do you have other suggestions on how I can verify that this config is active? If I pass in --module, --class or --file then those arguments will take precedence.
Comment #10
mondrake@jonathan1055 the way core executes individual PHPUnit testsuites is via run-tests.sh
--types "$TESTSUITE" --allarguments.But there is a BUT: there's a difference between the way testsuites are indicated in the .gitlab-ci.yml file and how they're defined in the phpunit.xml.dist testsuites section. run-tests.sh manages on the fly the conversion between e.g.
PHPUnit-Unit(in GitLab CI) andunit(in phpunit.xml) and viceversa.I do not remember what would happen if the
--types "$TESTSUITE" --allhas a $TESTSUITE value which is not one of the core phpunit's testsuites. Worth checking it, and if it fails, fix here to let fallback the conversion to the input value.I think it would be worth deprecating
--typesand--allarguments and introduce sth along the lines of--phpunit-testsuiteand--phpunit-groupto match PHPUnit's--testsuiteand--grouparguments, but IMHO should be part of a follow-up issue.Comment #11
needs-review-queue-bot commentedThe Needs Review Queue Bot tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".
This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.
Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.
Comment #12
mondrakerebased
Comment #13
mondrakeI will rebase the MR
Comment #14
mondrakeComment #15
smustgrave commentedRebase seems good.
Comment #16
catchOne question on the MR.
Comment #17
mondrakeComment #18
jonathan1055 commentedThanks for the hint @mondrake re #9 - #10. I have now sucessfully used this MR in a contrib pipeline. In my
custom.xmlI defined<testsuite name="only-default-time">then added--types only-default-time --all --phpunit-configuration $PHPUNIT_CONFIGURATION_FILE_PATHand alsoPHPUNIT_CONFIGURATION_FILE_PATH: $CI_PROJECT_DIR/custom.xmlin .gitlab-ci.yml and the specified subset of tests was run. This demonstrates that my specifiedcustom.xmlconfig must be active and that therefore we can use this mechanism in Gitlab Templates for Contib pipelines too.Comment #19
catchThis looks good to me but I agree with #5 that it could use a CR (could have been moved to needs work for that instead of RTBC).
Comment #20
mondrakeAdded draft CR
Comment #21
catchThis looks good to me, should make it easier long term for phpunit 12 etc.
Committed/pushed to 11.x, thanks!