[meta] Support PHPUnit 10 in Drupal 11

Thanks @longwave for creating this meta and doing all the housekeeping of the related issues.

A very first patch trying to load PHPUnit 10 dev to see what breaks. All sort of tricks in here :)

First insights:

• Listeners are gone, so for instance the deprecation bridge is gone too
• --printer option for cli is gone, so the custom HTML output is gone too
• Cannot composer require phpspec/prophecy-phpunit at the moment, too many constraint conflicts

With an up-to-date phpunit.xml.dist

Revived #3130606: MockBuilder::setMethods is deprecated in PHPUnit8 and removed from PHPUnit10 which becomes very relevant for PHPUnit 10.

MockBuilder::withConsecutive() method has been deprecated in PHPUnit 10.

Bumping to Drupal 10. Maybe in the future it will need backporting to D9, but D10 comes first.

An update.

Reroll and some cleaning

#20 looks like it's not necessary, the new event system can be exploited already. #31 does that, implementing a logger that tracks HTML output and prints it out at the end of the test run, subscribing to TestRunner events. Not touching anything around deprecation listeners yet, I think we need to see what will Symfony PHPUnit-Bridge will do first.

Some cleanup, introducing a singleton for the logger and making better use of the inners of DefaultResultPrinter for actual logging. Dropped the SimpletestUIPrinter logic since we no longer need to convert links to href tags for use in Simpletest UI.

More work around HtmlOutputLogger.

PHPUnit's new event system is really cool. I started implementing also a singleton for checking Drupal standards - this replaces DrupalComponentTestListenerTrait for instance.

#3254723: Remove SimpletestUiPrinter landed. Rerolling.

Adding the below to the IS:

Upstream, https://github.com/symfony/symfony/issues/45223 and https://github.com/symfony/symfony/pull/45226 would potentially allow skipping Drupal's implementation of DeprecationListener that allows silencing deprecations.

Allowing DrupalCI to execute PHPUnit against different phpunit.xml.dist, depending on the PHPUnit version being run, would allow minimizing the warning messages sent out when, for instance, PHPUnit 10 finds a listener section that is a must have in PHPUnit 9. For the moment it's just noise-avoidance, but who knows in the future.

Still unclear what

Obviously, wait for PHPUnit 10 to be a thing

means from a timeline perspective.

https://www.kiratas.com/unit-testing-framework-phpunit-10-requires-versi...

Anyway, Symfony's PHPUnit-Bridge will also have to be adjusted before we can tackle anything substantial here.

NEW!

PHPUnit\Framework\TestCase::addWarning() was made final so we can not override it any longer in Drupal\Tests\Traits\ PhpUnitWarnings.

Also,

Fatal error: Cannot override final method PHPUnit\Framework\TestCase::expectDeprecation() in /var/www/html/vendor/symfony/phpunit-bridge/Legacy/ExpectDeprecationTraitForV8_4.php on line 22

means that Symfony's PHPUnitBridge must be adjusted in order to manage deprecations.

PHPUnit\TextUI\DefaultResultPrinter is also removed so we need a new solution for the HtmlOutputLogger.

Why a blocker? PHPUnit 9 currently tests fine on PHP 8.2, https://github.com/sebastianbergmann/phpunit/actions/runs/2523925481, I doubt it will drop its support when PHPUnit 10 will be released (unknown timeline ATM, see #56 and #40).

Also - a release of PHPUnit 10 will NOT be sufficient - we will have to wait for Symfony's PHPUnit-Bridge to adjust to PHPUnit 10 (whatever it will mean, I cannot find prep issues upstream) before we can tackle Drupal's own stuff.

This is basically a "beta blocker until it's not" -- the PHPUnit 10 milestone currently reports 90% complete. I check it once every couple of weeks. :) So we should continue to try to solve PHPUnit 10 compatibility problems in D10 to the best of our ability and support/require it if it's ready in time.

And if it's not, well, we tried. :)

Just to be clear, we won't delay D10 if PHPUnit 10 still is not available at all by the scheduled window and if other requirements are complete -- but it's not quite in the same category as the various "should-have" issues, because those can be deferred to D11 more gracefully than PHPUnit. It almost needs its own special tag, but then no one would find it to monitor it. ;)

Clear. Thanks @xjm.

Changed to MR workflow and trying to re-enable HTML debug logger.

rebased

#3306554: InvocationMocker::withConsecutive() is deprecated in PHPUnit 9.6 and removed from PHPUnit 10 is going to have a PHPUnit deprecation in PHPUnit 9.6, https://github.com/sebastianbergmann/phpunit/issues/5035

This now becomes a Drupal 11 requirement.

Thanks @mondrake!

Re-parenting to the correct meta (we can do this in D10 minors).

Symfony have started the discussion on how to adjust the PHPUnit-Bridge package for PHPUnit 10, https://github.com/symfony/symfony/issues/49069.

prophecy-phpunit too, https://github.com/phpspec/prophecy-phpunit/pull/41

Updated IS.

PHPUnit 10 will be released on February 3, 2023, in two days as per https://thephpcc.social/@sebastian/109788385318681908

Updated issue summary with this info.

Hat tip to @larowlan for the info.

Link to changelog

Concurrently with PHPUnit 10, PHPUnit 9.6 will be released, too. That release will deprecate quite a number of things that we also need to work on before we can tackle PHPUnit 10: https://www.drupal.org/project/issues/search?issue_tags=PHPUnit%209.6

PHPUnit 10 is out, https://phpunit.de/announcements/phpunit-10.html

An annoucement worth reading.

Fair bit going on in that post, thanks for sharing.
.
We're going to need to wait for prophecy and the symfony bridge to have support too

Targeting D11 sounds like a reasonable plan

If PHPUnit release schedule goes back to yearly as it was in the past, https://github.com/sebastianbergmann/phpunit/milestones, PHPUnit will be on its 11 release by the time D11 ships. And PHPUnit 10 will likely be EOL in Feb 2025, while D11 will be still on. Combined with PHP evolution itself, PHPUnit 10 may become obsolete during D11 lifecycle. Fun.

I feel like the issue title is no longer representing what's being discussed here, I suggest to revert to an updated version of the prior one.

Added reference to https://github.com/sebastianbergmann/phpunit/issues/5201 in the IS

rebase

phpspec/prophecy-phpunit 2.1.0, together with phpspec/prophecy 1.18.0, are meant to be PHPUnit 10 compatible

The MR now runs all the test in PHPUnit 10, so we can track impact when the related items fix things.

Some fixes after #3362083: '@requires externalCommand' is not parsed in PHPUnit 10.

Prophecy have PHPUnit 10 compatibility fixes in, but have not released yet.

Symfony PHPUnit-Bridge is rather silent, I have opened a PR https://github.com/symfony/symfony/pull/50371, its diff is used to get the MR here going. But that is mostly experimental.

Fixing attribution.

Don't see any discussion on it but our test suite registration is no longer supported in PHPUnit 10.

https://github.com/sebastianbergmann/phpunit/commit/36354a9cdc69ecabd20a...

Might be a really cool solution in the current patch I'm missing though.

#95 since there's absolutely nothing in the current patch that thoughtfully addresses test suites, I started looking for miracles... :)

But ended up, in my understanding, finding that we are not 'using' test suites at all even now. AFAICS. run-tests.sh builds list of individual tests to be run in parallel, but does not use test suites (in PHPUnit's sense) for that, rather our own-baked definition of test groups via matching test classes namespaces. It does so in TestDiscovery::getPhpunitTestSuite().

Since core is only using run-tests.sh, I wonder what are we defining TestSuites for? Just for those that want to use the suite filtering when running PHPUnit via its own CLI? If that's the case then I think we should drop this - at least start deprecating them now.

#3402444: Deprecate Test Suites, no longer available in PHPUnit 10 shows IMHO that test suites are just cruft currently.

Re: #98 failures.

It looks like #3387737: Split PHP image into php(cli/apache) and yarn(node/nightwatch) removed the patch command from images.

Spent a bunch of time digging into the discovery process related to the test suites and that sent me down a lot of rabbit holes. The test suite part I did not enjoy but it ended with me having a mostly working unit test suite so that's great! Of course mondrake was ahead of me on basically all of those changes.

Tried to capture all of my findings and the various work happening in an updated IS.

Will capture some issues for some of the things I found that I think can be addressed in the short term.

Great writeup, thanks @neclimdul!

Hide patch file, try to get this into the MR

Found another interesting issue exposed by the PHPUnit 10 updates.

#3406024: DependencySerializationTrait depends on removed __PHPUNIT_BOOTSTRAP global

The correct fix probably needs some architecture discussion because it would require changes to DependencySerializationTrait which affects almost everything.

phpspec/prophecy-phpunit 2.1.0 is now in

Opened #3417066: Upgrade PHPUnit to 10, drop Symfony PHPUnit-bridge dependency to discuss removal of Symfony's PHPUnit bridge in case it is not getting compatible with PHPUnit 10 in time for Drupal 11 release.

Amazing work here :) It would be great to clean up the issue summary to current priorities and remaining work. I am not sure I can tell in each point whether its done or not. For example the Symfony PHPUnit bridge question is handled in #3417066: Upgrade PHPUnit to 10, drop Symfony PHPUnit-bridge dependency. This seems to be the biggest backend dependency update for Drupal 11 apart of Symfony 7 and it would be great to have this in place sooner than later so that contrib modules can update to and test with the new version.

Also as per the comments above PHPUnit 11 is about to be released later this week on Friday. As per https://phpunit.de/supported-versions.html that also means end of bugfix support for PHPUnit 9 that we are using now. While PHPUnit 9 and 10 EOLs have not been determined yet (again as per that page), people argued above that bugfix support for PHPUnit 10 will also end in 2025 February, early into the Drupal 11 release cycle. So if at all possible, we should get to PHPUnit 11 in time for Drupal 11 :) (not in this issue, but this is a prerequisite).

Is there any way the committer team can better help? Is framework manager review needed on certain areas? Should the issue be broken out to (further) smaller changes?

Opened #3418267: Support PHPUnit 11 in Drupal 11 as postponed and put it in the issue summary as a followup :) That allows us to track it on #3267879: [meta] Add compatibility for the latest major and minor versions of dependencies to Drupal 10 too.

Updated the IS with the completed tasks.

Let’s be honest.

@mondrake Per https://www.drupal.org/blog/drupal-10-will-be-supported-until-the-releas... there will still be Drupal 10.3, 10.4 and 10.5, and PHPUnit 10/11 support is something that would be backported if the extent of the allowed changes allow it, because it supports contributed projects to be compatible with multiple Drupal major version. Is it not possible to target this at Drupal 10 even in that light? Especially of question I think is Drupal 10.3 which will be in beta in a little over 3 weeks from now per https://www.drupal.org/about/core/policies/core-release-cycles/schedule#...

Re #123 honestly, I have doubts it would be possible to backport, given the amplitude of the changes from PHPUnit 9 to PHPUnit 10.

I still see at risk to close all the open issues by D11, frankly. Then, another significant effort will be needed for supporting PHPUnit 11: https://github.com/sebastianbergmann/phpunit/milestone/46?closed=1. And we will be late at that again, with PHPUnit 12 https://github.com/sebastianbergmann/phpunit/milestone/52 being 3 to 6 months away at that point.

PHPUnit major releases timeline, being yearly, is unforgiving for frameworks like Drupal that is much more focused on keeping BC, unfortunately. It's a philosophical question, at the end.

Eager to read others' point of view here.

I think the most important thing is whether modules can write phpunit 10 ( and eventually 11) compatible tests against Drupal 10, so that they don't need to maintain two branches. The actual version of phpunit that those tests run against is secondary.

Beyond that, it would be nice if we could widen Drupal 10's phpunit constraint to allow phpunit 10, but seems like that will be useless if we have a hard requirement on symfony phpunit bridge. That then brings up whether we want to do some Drupal 10 surgery to make symfony phpunit bridge an optional requirement (fork it? put it behind a metapackage so it's skipped when phpunit 11 is installed?), I don't think it's worth spending time on that until we know what we're doing in 11.x fully, since that'll determine exactly how much gap needs to be bridged.

Actually updating to phpunit 10 and dropping phpunit 9 support seems out of the question due to the scale of changes we're having to do in core.

So far, nearly all the changes we've been committing to core for phpunit 10 have been to stop using deprecated features, and we've been able to backport those to Drupal 10 without any issues, and there is nothing stopping contrib modules from making those same changes. But error and deprecation handling are still @todo.

The big issue with phpunit 11, is will it be dropping support for phpunit 9 features that were only added in phpunit 10? If so, then Drupal 10 will be stuck unless we try to add forwards compatibility shims, this is also too far beyond where we currently are to worry about now I think.

Then the last issue is whether phpunit 9 is going to support the php versions that Drupal 10 is going to need to support, that's also going to be a pain if it doesn't, although it's unlikely to be the only dependency with that problem.

PHPUnit 11.0 is already out. PHPUnit works by soft-deprecating in one version, hard-deprecating in the next, and removing in the one after, so anything soft-deprecated in 9 will be removed in 11

Forgot that detail - that's a good point so we should be fine split between phpunit 9 and 10 and just recommend people deal with all phpunit 9 deprecations.

Few updates to the IS.

We should update PHPUnit as https://github.com/sebastianbergmann/resource-operations is also an archived library and is required only on the last version supported by Drupal 9.6.17 https://packagist.org/packages/phpunit/phpunit#9.6.17

+1

We're down to the 2 last issues to be committed, #3400366: Remove DrupalComponentTestListenerTrait and replace with a PHPStan rule and #3417066: Upgrade PHPUnit to 10, drop Symfony PHPUnit-bridge dependency, then this one would be done - 3 years and 160 issues later :)

PHPUnit 10 is done! I wonder if we want to keep this open to track issues with the change record and/or discovered by contrib modules once thy start testing against it (or a new issue to track that?) otherwise I think we can close this too.

Seems like this just broke the current D11 (next major) Gitlab template. See e.g. https://git.drupalcode.org/issue/htmx-3447092/-/jobs/1591833

The problem is the use of the --no-interaction CLI flag, which no longer exists in 10.5.20

@fgm see this for the template adjustments: #3444792: Prepare for PHPUnit 10.

This looks done to me. Feel free to reopen if we want somewhere to track stuff, but moving it away from RTBC to reflect state of queue.