Problem/Motivation

In converting PHPUnit test metadata annotations from annotation to attributes, we renamed most of the @covers annotations to @legacy-covers, since @covers has not a direct equivalent in attributes.

Also, cover attributes are no longer supported on the test methods, only as class targets.

This is leaving a significant amount of cruft @legacy-covers annotation in the codebase, and more so there are new additions that do not make sense as they will unlikely be changed to something meaningful, at least in bulk.

Proposed resolution

Add @legacy-covers to the list of forbidden annotations on test methods, baseline existing instances, let the future fix the history.

Remaining tasks

User interface changes

Introduced terminology

API changes

Data model changes

Release notes snippet

Issue fork drupal-3555692

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

mondrake created an issue. See original summary.

mondrake’s picture

Uh oh... 4185 errors would be added to the baseline...

longwave’s picture

Not sure this worth doing, why would we add new instances of this annotation now the conversion is done?

mondrake’s picture

Priority: Normal » Minor

We shouldn't, but I saw this happening in a commit recently (sorry I lost track of which one). But yeah - it's harmless anyway and now numbers are just too big to baseline them at the moment.

mondrake’s picture

longwave’s picture

Status: Active » Needs work

Okay, let's try to do it - but what should we do about the existing instances? Should we just delete them if they are meaningless?

mondrake’s picture

Let's discuss #7 first.

The options I see:

1) we just remove any @legacy-covers through a Rector rule. Easy, but we may lose meaningful information.

2) we convert @legacy-covers to a generic comment through a Rector rule. Example;

from

  /**
   * @legacy-covers ::buildProxyClassName
   */
  public function testBuildProxyClassName(): void {

to

  /**
   * Tests the ::buildProxyClassName() method.
   */
  public function testBuildProxyClassName(): void {

Easy, but could lead to silly docblocks like e.g.

from

  /**
   * Tests set form state.
   *
   * @legacy-covers ::setFormState
   */
  public function testSetFormState(): void {

to

  /**
   * Tests set form state.
   *
   * Tests the :setFormState() method.
   */
  public function testSetFormState(): void {

3) We bring all the @legacy-cover entries defined on the test methods to the test class as a #[CoversMethod()] attribute. Relatively easy again, but would end up in a messy combination of #[CoversClass] and #[CoversMethod] attributes on the class level, and coverage declaration will not be relevant any more on the paired runtime-method<->test-method.

4) We split all test classes to test only one method each. Crazy difficult.

We also need to decide what we do when we have multiple @legacy-covers in the same docBlock.

longwave’s picture

I've just re-read the discussions on GitHub and it's somewhat annoying that multiple people have requested CoversMethod to be allowed on test methods but PHPUnit have decided that this is not necessary, without any real explanation other than "it's not needed". They seem to imply that #8.4 is the way forward but there's no way we can implement that in Drupal given the size of the existing test suite. But if they are insistent on not adding the feature then I am not entirely sure of the point of keeping this metadata in the codebase, because we won't be using it for anything either.

I think that we could remove @legacy-covers in cases where the test method name starts with the covered method, e.g. in OverridesSectionStorageTest:

 * @legacy-covers ::getContexts
 */
public function testGetContexts(): void {
...
 * @legacy-covers ::getContextsDuringPreview
 */
public function testGetContextsDuringPreview(): void {
...
 * @legacy-covers ::getDefaultSectionStorage
 */
public function testGetDefaultSectionStorage(): void {
...
 * @legacy-covers ::getTempstoreKey
 */
public function testGetTempstoreKey(): void {

or BlockComponentRenderArrayTest:

 * @legacy-covers ::onBuildRender
 */
public function testOnBuildRenderInPreviewEmptyBuild(): void {

 * @legacy-covers ::onBuildRender
 */
public function testOnBuildRenderEmptyBuild(): void {

 * @legacy-covers ::onBuildRender
 */
public function testOnBuildRenderEmptyBuildWithCacheTags(): void {

To me this isn't adding much value.

Hoisting them all to the class level also seems pointless when our test classes are often 1:1 with the classes under test - I think you will end up with a list of all/most public methods in the class, which again doesn't add value. But in cases where this isn't true, perhaps there is value in hoisting them to the top level - not sure how many cases we will have of that though.

mondrake’s picture

Status: Needs work » Postponed

#9 sounds like a good first step, let's do that and see how the numbers will change. Opened a separate issue #3557840: Remove @legacy-covers in cases where the test method name starts with the covered method for that.

mondrake’s picture

Priority: Minor » Major
Status: Postponed » Active

I have seen a couple of commits in the last days that added more @legacy-covers annotations. Despite numbers, I think we should stop the bleeding, add the rule, and baseline the errors. #3557840: Remove @legacy-covers in cases where the test method name starts with the covered method would eliminate approx 1k instances.

mondrake’s picture

Status: Active » Needs review
smustgrave’s picture

Status: Needs review » Reviewed & tested by the community

Agree putting the blocker would be nice

catch’s picture

Status: Reviewed & tested by the community » Needs work

Sorry I don't think we should baseline this unless we know what we're going to do about them, for a start it means there's no way to remove them from the baseline consistently.

I've committed MRs with @legacy-covers recently because that's what we used for existing annotations in core when we updated, on the basis that we would convert them from @legacy-covers to something else later but that the information would be preserved in the meantime.

If we're just going to delete them, we should do that, and then add the phpstan rule. If we're going to do something else, we should figure out what that thing is going to be first.

mondrake’s picture

Status: Needs work » Postponed
longwave’s picture

If we think this information is valuable but we want to get rid of these annotations, there is nothing stopping us adding CoversMethod in our own namespace (or even stealing PHPUnit's namespace, but that feels very wrong). Then if we do more with the coverage metadata in the future then we can read the annotation ourselves if needed, or if PHPUnit have a change of heart then we just have to swap back to their namespace. I am still on the fence as to whether this is useful at all in many cases, though.

mondrake’s picture

Yep. @covers annotations in the docblock will keep throwing a PHPUnit deprecation until when we bump PHPUnit to 12. At that point, PHPUnit has just removed the annotation parsing code, and we theoretically could revert @legacy-covers to @covers, if we wish. But that would be confusing IMO - PHPUnit will do absolutely nothing with it, and do not know what we would be doing with that on our own, given that coverage reporting is produced by PHPUnit anyway.

In the meantime, #3557840: Remove @legacy-covers in cases where the test method name starts with the covered method would get rid of approx 25% of the problem.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.