Problem/Motivation

Core's phpunit.xml.dist PHPUnit configuration file flashes some warnings during test discovery under PHPUnit 11, as it keeps supporting PHPUnit 10 directory patterns.

Proposed resolution

Remove the PHPUnit 10 specific configuration while keeping support for PHPUnit 10 for contrib if they need to downgrade.

Remaining tasks

User interface changes

Introduced terminology

API changes

Data model changes

Release notes snippet

Issue fork drupal-3527934

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

mondrake created an issue. See original summary.

mondrake’s picture

Issue tags: +PHPUnit 11, +PHPUnit 12
mondrake’s picture

Title: Remove support for PHPUnit 10 » Optimize phpunit.xml.dist for PHPUnit 11 while retaining support for PHPUnit 10
Issue summary: View changes

mondrake’s picture

Status: Postponed » Needs review
catch’s picture

Tagging for 11.3.0 release priority/release notes.

I think if we remove support for running core tests with phpunit 10 but allow it for custom CI pipelines etc., that's a good compromise. The warnings here are currently causing performance tests jobs on 11.2.x to fail with phpunit warnings.

xjm’s picture

@catch and I discussed this and we think it's probably safe to do in 11.3 as a bridge toward using PHPUnit 12 (or even 13) in D12, so long as we retain the open constraint allowing PHPUnit 10 for contrib/custom CI that hasn't had a chance to update. As @catch stated:

The worst disruption would be if you're running without customising anything in phpunit.xml in a custom CI, you might need to customise it. And the core test suite as a whole wouldn't run against phpunit 10 any more.

The two reasons not to wait are:

  1. our phpunit.xml is not fully compatible with phpunit 11, and we don't have support (on ci, in run-tests.sh) for switching between different ones yet.
  2. phpunit 12 was released three months ago and phpunit 13 will be out before Drupal 12 is released

That said, we'd like to re-evaluate this once it's ready for commit just to double-check that there's no additional disruption nor aspects that we missed, so adding the needs RM review tag for that.

I wonder if a CR is enough of a notification about this change, or if we should do a core blog post or something? In the olden days this would have been the sort of thing we announced ahead of time on g.d.o/core but the core blog is a bit more (perhaps too) visible for that. Another thing to consider in our final RM review above.

Anyway great issue! Thank you.

catch’s picture

Status: Needs review » Needs work
Issue tags: +Needs change record

I reviewed the issue, kicked off a pgsql and sqlite MR just in case, and ran the phpstan tests optional job, and I can't really see anything to complain about here at all - except that we need a change record and release note. So marking needs work for those, but from the MR perspective looks really good to me.

mondrake’s picture

While pondering on the CR to write, I realized we can probably do better here. If we do #3530113: Allow indicating alternative phpunit.xml than core's when testing via run-tests.sh first, then we would be able to add here a PHPUnit 10 version of the phpunit.xml (i.e. a copy of the current phpunit.xml.dist in HEAD), and in the CR recommend contrib to switch to use it when running D11 tests with PHPUnit 10. Much neater and less instructions required to tell how to change the xml.

In practice, we would swap the current state where phpunit.xml.dist supports PHPUnit 10 (and 11 with the warnings triggered), and .phpunit-next.xml is the PHPUnit 11 only, currently used for testing only. We would end up with a phpunit.xml.dist for PHPUnit 11 only, and a .phpunit-10.xml for PHPUnit 10 only.

mondrake’s picture

Status: Postponed » Closed (outdated)

#3530113: Allow indicating alternative phpunit.xml than core's when testing via run-tests.sh has taken over here. We need an issue to remove PHPUnit 10 support, but let's create a new one instead of reverting this one to its early state.