Problem/Motivation

From #3217904-95: [meta] Support PHPUnit 10 in Drupal 11.

PHPUnit 10 drops support for the 'test suites' concept. https://github.com/sebastianbergmann/phpunit/commit/36354a9cdc69ecabd20a...

original issue that drove the requirements for this discovery mechanism.
#2499239: Use test suite classes to discover different test types under phpunit, allow contrib harmony with run-tests

Proposed resolution

Deprecate own implementation of test suites.

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

CommentFileSizeAuthor
#43 3402444-nr-bot.txt90 bytesneeds-review-queue-bot

Issue fork drupal-3402444

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

mondrake created an issue. See original summary.

mondrake’s picture

Status: Active » Needs review

The deprecations only affect the test of the feature itself.

spokje’s picture

Status: Needs review » Reviewed & tested by the community

Left a question in the comment, but I think this is ready for inspection by the core committers.

neclimdul’s picture

I'm not sure I agree with the understanding of the issue.

1. run-tests is _one_ way to run tests against Drupal. The existence and documentation of the use of core/phpunit.xml.dist is the other. Run-tests is so clunky, awkward, and slow I haven't used it in 10 years or more at this point? It works for testbot but I can't figure out how anyone would use it regularly.
2. moving away from exposing our tests to phpunit blocks deeper integration into the phpunit ecosystem which I feel like is _not_ where we want to be heading. Not running tests in your IDE natively, not being able to look at paratest, etc.

So we don't need to deprecated it, we need to replace it.

neclimdul’s picture

Status: Reviewed & tested by the community » Needs work
mondrake’s picture

Status: Needs work » Needs review

#5. Let me try clarifying since it's not clear.

So the replacement in PHPUnit 10, independently from run-tests.sh, would be to remove the cruft implementation classes and replace (if possible) with a direct indication of the directories to scan in phpunit.xml.dist.
Hence deprecation here.

neclimdul’s picture

Status: Needs review » Needs work

We originally tried to do this in directory definitions but eventually had to give up because it was so dynamic and varied with the structures Drupal can exist with. PHPUnit is also pretty limited in how you can define these. I don't think that requirement hasn't really changed so I don't think we can deprecate without a replacement.

I don't think this is a problem though, if we're clever, we can probably move the logic into the events available in PHPUnit 9+ with probably opens up a lot better functionality and growth. We should even be able to do a deprecation in Drupal 10 without converting fully to PHPUnit 10. Just triggered if the static method is called directory and then pointing users to update their local configuration along the timelines of the PHPUnit 10 to what ever the event system looks like.

mondrake’s picture

Time will tell. Anyway we agree these classes will have to go away, right?

neclimdul’s picture

Issue summary: View changes

So yeah, I was hopeful about the event and extensions but after tracing up and down there's not much to be done. The classes need to go away, just not sure what the path forward for supporting our various testing logic. Mostly discovering tests in submodules and contrib.

Added the original that added discovery logic to summary.

mondrake’s picture

Status: Needs work » Needs review

MR still applies, up for review again then

spokje’s picture

Status: Needs review » Reviewed & tested by the community
longwave’s picture

Status: Reviewed & tested by the community » Needs work
Issue tags: +Needs change record

The deprecations need a change record to point to.

mondrake’s picture

mondrake’s picture

Status: Needs work » Reviewed & tested by the community

Adjusted references to CR and rebased.

neclimdul’s picture

I'm not sure I was being clear since this is back to RTBC but my concerns aren't addressed.

Because of the way this is structured and the way the message is worded this looks like we're deprecating phpunit entirely.

Remaining self deprecation notices (2)

  1x: \Drupal\Tests\TestSuites\UnitTestSuite is deprecated in drupal:10.3.0 and is removed from drupal:11.0.0. There is no replacement. See https://www.drupal.org/node/3405829

  1x: Drupal\Tests\TestSuites\TestSuiteBase is deprecated in drupal:10.3.0 and is removed from drupal:11.0.0. There is no replacement. See https://www.drupal.org/node/3405829
    1x in UnitTestSuite::suite from Drupal\Tests\TestSuites

The problem is, there's no replacement for the class, but we should have a replacement in PHPUnit 10 for running PHPUnit tests.

I was asking that we figure out what that replacement is, then provide that explanation in the message and CR instead of just saying there's no replacement.

mondrake’s picture

Status: Reviewed & tested by the community » Needs work

NW then.

neclimdul’s picture

Maybe to limit the scope, keep this clean, and not build a circular dependency between this and the larger PHPUnit upgrade we can do s/There is no replacement/There is no replacement and test discovery will be handled differently in PHPUnit 10./ and then the CR has some block about following the PHPUnit 10 upgrade issue. At least if that's ok with committers.

We can then clarify everything when finalize the upgrade path.

catch’s picture

I think #18 is OK, the change record can point out this doesn't affect running individual tests locally (or run-tests.sh on gitlab) etc.

Do we need a new follow-up issue to figure out what the replacement is or is one already open? There's quite a few notes in the issue summary of #3217904: [meta] Support PHPUnit 10 in Drupal 11 which could be copied over along with findings from here.

mondrake’s picture

Status: Needs work » Needs review

Changed deprecation message.

smustgrave’s picture

Status: Needs review » Needs work

Seems to have phpcs failures.

mondrake’s picture

Spell-checking… I cannot explain the job failure though, seems related to git branches, not to real spelling errors

longwave’s picture

Rebase the branch against 11.x; the spellchecking errors have been solved in the core repo but GitLab CI runs the scripts from when the MR was branched.

spokje’s picture

Just did that ^
:)

longwave’s picture

Status: Needs work » Needs review
spokje’s picture

All green now (as @longwave already predicted above apparently ;)

mondrake’s picture

Thanks @Spokje! Mind sharing what commands did you use to get that?

spokje’s picture

The rebase button/link has disappeared in MRs, but just adding /rebase as a comment on the MR (so in this case: https://git.drupalcode.org/project/drupal/-/merge_requests/5452) will auto-rebase if there aren't any conflicts that GitLab can't solve without manual intervention.

If the latter is the case, the command will do nothing.

mondrake’s picture

TIL^^

Thanks!!

smustgrave’s picture

Status: Needs review » Reviewed & tested by the community

I also was not aware of that thanks!

I believe feedback has been addressed though

spokje’s picture

Note on the TIL: You must have push-access on the MR for the magic to work.

longwave’s picture

Status: Reviewed & tested by the community » Fixed

Committed dccd322 and pushed to 11.x. Thanks!

Also published the change record. See you all in the next PHPUnit 10 issue!

  • longwave committed dccd3221 on 11.x
    Issue #3402444 by mondrake, Spokje, neclimdul, catch: Deprecate Test...

  • catch committed 33630ea5 on 11.x
    Revert "Issue #3402444 by mondrake, Spokje, neclimdul, catch: Deprecate...
catch’s picture

Status: Fixed » Needs work

This broke the performance test job on gitlab: https://git.drupalcode.org/project/drupal/-/jobs/788445 Not sure exactly why yet but reverted for now.

mondrake’s picture

I think that’s because those tests are run directly via Phpunit CLI —group and not via run-tests.sh.

That triggers PHPUnit’s own test discovery that loads all the classes and therefore triggers the deprecation errors. run-tests uses its own test discovery and then runs test on a file-by-file basis, so that doesn’t occur.

Not sure what is the best way forward. Maybe just avoid the @trigger_error of the deprecation of the classes so we don’t get this issue with autoloading.

mondrake’s picture

Status: Needs work » Needs review

This could do, I think.

We remove @trigger_error on classes, that fires on class autoloading, leaving the @deprecate annotations. That's enough to have PHPStan detect usage of deprecated classes, in case - that's demonstrated by the fact that we need to add to PHPStan baseline the extension of TestSuiteBase by its child classes.

quietone’s picture

I've unpublished the CR

smustgrave’s picture

So what was the solution for the problem seen in #36? Or still being determined?

mondrake’s picture

#37 is the explanation, and #39 is the solution. Is that not clear enough?

needs-review-queue-bot’s picture

Status: Needs review » Needs work
StatusFileSize
new90 bytes

The Needs Review Queue Bot tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".

This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

mondrake’s picture

Status: Needs work » Needs review

Come on, bot…

catch’s picture

Status: Needs review » Reviewed & tested by the community

Looks good to me, will commit later if no comments since this already landed once.

spokje’s picture

Issue tags: +no-needs-review-bot
catch’s picture

Status: Reviewed & tested by the community » Needs work

Needs a rebase.

mondrake’s picture

Status: Needs work » Reviewed & tested by the community

rebased, no clashes

  • catch committed d176ccd2 on 10.3.x
    Issue #3402444 by mondrake, Spokje, neclimdul, longwave: Deprecate Test...

  • catch committed 79e8ae5d on 11.x
    Issue #3402444 by mondrake, Spokje, neclimdul, longwave: Deprecate Test...
catch’s picture

Version: 11.x-dev » 10.3.x-dev
Status: Reviewed & tested by the community » Fixed

Committed/pushed to 11.x and 10.3.x, thanks!

mondrake’s picture

Published the CR. As to ‘what now, then?’, the MR in #3417066: Upgrade PHPUnit to 10, drop Symfony PHPUnit-bridge dependency has now a proposal to include the files to be included in the suites, that covers core lib tests, core module tests, and tests for modules that are placed in the /module subdirectory. I think the CR will have to be updated once that eill be confirmed to be the final approach for D11/PHPUnit 10.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.