Problem/Motivation

Implementing strict typing in existing code is challenging. If we don't enforce it on new code we'll constantly be chasing our tail. We get missing type checks out of the box with phpstan level 6, but we're currently stuck on level 1.

This issue proposes enabling just the rules required to enforce return types, while remaining at the current level.

Steps to reproduce

N/A

Proposed resolution

  • Enable MissingFunctionReturnTypehintRule and MissingMethodReturnTypehintRule phpstan rules
  • Ignore missingType.iterableValue
  • Update the phpstan baseline to include all violations of this rule

Remaining tasks

Review the changes to phpstan.neon.dist and what's currently being added to the baseline, even if it's not mergeable, to verify that we're only adding violations of identifier: missingType.return.

For example:

# Check how many new errors are ignored.
$ git diff 11.x | grep '^\+$ignoreErrors\[]' | wc -l
12378

# Check how many 'missingType.return' errors are added.
$ git diff 11.x | grep '+.*// identifier:' | grep 'missingType.return' | wc -l
12378

# Check what other lines are removed.
$ git diff 11.x | grep '^\-\s'
-	// identifier: variable.undefined
-	'message' => '#^Variable \\$patterns might not be defined\\.$#',
-	'message' => '#^Missing cache backend declaration for performance\\.$#',
-	// identifier: variable.undefined
-	'message' => '#^Variable \\$inner_count might not be defined\\.$#',

Baseline size comparison

Before: 2640 lines, 115K
After: 76908 lines, 3.7M

User interface changes

API changes

Data model changes

Release notes snippet

All new functions and methods must have return types defined.

CommentFileSizeAuthor
#7 3461318-nr-bot.txt90 bytesneeds-review-queue-bot

Issue fork drupal-3461318

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

mstrelan created an issue. See original summary.

mstrelan’s picture

Status: Active » Needs review
andypost’s picture

Stats for baseline looks big but looks like it should be better then growing amount of newcode

+ 75718
− 1390
smustgrave’s picture

Status: Needs review » Needs work

MR appears to be unmergable

mstrelan’s picture

Status: Needs work » Needs review

Obviously it will need constant rebasing. Setting back to NR to get actual review.

needs-review-queue-bot’s picture

Status: Needs review » Needs work
StatusFileSize
new90 bytes

The Needs Review Queue Bot tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".

This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

mstrelan’s picture

Issue summary: View changes
Status: Needs work » Needs review
Issue tags: +no-needs-review-bot

Let's just review the idea and the changes to phpstan.neon.dist. If we're happy with that then we can regenerate the baseline. We can also review what's currently being added to the baseline, even if it's not mergeable, to verify that we're only identifier: missingType.return.

smustgrave’s picture

Enable MissingFunctionReturnTypehintRule and MissingMethodReturnTypehintRule phpstan rules

What if we broke these out into 2 tickets would that be easier?

mstrelan’s picture

$ grep "Method" 8763.diff | wc -l
10753
$ grep "Function" 8763.diff | wc -l
7335

Not sure there's any difference reviewing 10000 or 17000 changes.

smustgrave’s picture

Would it be easier to breakout into groups of components or group of modules?

Just brainstorming to help keep the idea moving.

mstrelan’s picture

I appreciate that the MR is massive but really the only change that needs review at all is to phpstan.neon.dist. Everything else is proven by the phpstan job passing. I don't think committing in chunks will help, this probably just needs "Needs ____ review" but I don't know what ____ is, maybe FM?

smustgrave’s picture

Framework would probably be best. And would just ping them.

quietone’s picture

Can you add the current size of the baseline and the size with this MR?

mstrelan’s picture

Issue summary: View changes
mstrelan’s picture

Rebased, re-baselined and rewrote the issue summary. FWIW I don't know why some lines were removed from the baseline, but that can only be a good thing as long as the phpstan job still passes.

smustgrave’s picture

So what would be the scope or game plan to address these eventually? So people aren’t doing a bunch of tickets

mstrelan’s picture

This issue is not about adding the return types to existing methods, it's just about preventing new methods from being added that don't have a return type. I think adding return types to methods in run time code needs BC consideration most of the time.

acbramley’s picture

Big +1 on this and agree with #12. The longer we wait to do this the larger the potential baseline as more and more code gets added, I know we're pretty good at asking for return types but this would enforce it.

spokje’s picture

*jumps on the +1 bandwagon*

I think adding return types to methods in run time code needs BC consideration most of the time.

Having tried to work on PHPStan lvl 1 issues in the current baseline in a previous life, I very much think this is the case.

I see a slight risk of people adding a new missingType.return error to the baseline, since they're seeing the current ones also being in the baseline, but with a explanatory CR and, since almost nobody will read that, a vigilant NR and core committers crew (so no problem there :) we will be fine.

smustgrave’s picture

Status: Needs review » Reviewed & tested by the community

Let’s see if it can get in

smustgrave’s picture

Also imagine if this gets in a number of tickets may get sent back

bbrala’s picture

This is a great step imo to start enforcing.

This might also need an update to the Coding Standards. And if this is enforced do we need to say something about new classes that might not be able to add since they for example implement an interface or extend a base class? There might be place where this is imossoble.

mstrelan’s picture

Due to covariance you can add a return type to a child class when the parent or interface doesn't have one. See https://www.php.net/manual/en/language.oop5.variance.php

catch’s picture

Status: Reviewed & tested by the community » Needs work

This needs another re-roll, I'm +1. Pinged the other committers to try to get sign-offs.

mstrelan’s picture

Status: Needs work » Needs review

Rebased and rebaselined. Will probably need to do again before commit, so here's the steps I took.

git reset --hard 96204beed3f
git rebase 11.x
composer install
vendor/bin/phpstan analyze --configuration=./core/phpstan.neon.dist --memory-limit=-1 --generate-baseline=core/.phpstan-baseline.php
git add core/.phpstan-baseline.php
git commit -m "Update baseline"
git push --force

catch credited larowlan.

catch credited longwave.

catch credited xjm.

catch’s picture

Discussed this briefly with @xjm, @longwave, @larowlan and @quietone - we're all +1 so I am removing the RM/FM tags.

@xjm suggested doing a core blog post announcing the change since potentially a lot of existing MRs in the queue would start failing on PHPStan (even if they hopefully would have been eventually marked needs work via manual review). I'm not sure this is necessary because this is already a coding standard, we're just adding it to phpstan.

Double checked and this is already clearly documented in the coding standards exactly how we'll be enforcing it in PHPStan https://www.drupal.org/docs/develop/standards/php/php-coding-standards#s...

andypost’s picture

Status: Needs review » Reviewed & tested by the community

It needs docs update and blog post is a good idea as well

I think it could be done after commit and will bring nice amount of rerolls for Barcelona's sprints

kingdutch’s picture

Although this does go against the PHPStan philosophy which is "Don't pick and choose which rules you apply, it works best if you apply all of them", at this point I'm happy to have any progress towards better type support in Drupal.

The only thing I'd want to add is that this doesn't change the need for the related issues mentioned in the parent. Even though it provides an important step in the right direction by adding return type hints for new code.

Also unrelated to the issue at hand but important to share with everyone in this thread vis-a-vis "how to add return types to interfaces": PHPStan is perfectly happy accepting a return type as PHPDoc as per the following example.

This means that as long as we force the rule to be enabled and downstream projects enable it then a viable strategy is:

1. Ensure all interfaces have a properly typed `@return` type annotation
2. In a major version promote the return type from PHPDoc to PHPCode.

PHPStan will already start reporting the return type mis-match based on the PHPDoc in downstream projects and help them implement it on classes (because as mentioned earlier in this thread, a class may always return a narrower type than what it extends/implements thanks to covariance). That means that as long as they fix those PHPStan errors before making the major jump, the major version jump won't actually be breaking for them. This aligns well with our current "Fix all deprecation on the last minor and then jump to the next major without changes" philosophy.

mondrake’s picture

The (minor) risk I see with this is that people start adding diverging types to class methods that inherit a common, yet untyped method interface. Covariance would allow do so now, but then it will make life more complicated later once we will try to typehint the interface.

See #3470913: Ensure getWidth()/getHeigth always return ?int and Symfony's upstream [ErrorHandler] Add support for @return-type-will-change #58134 for a different (but certainly longer) approach.

kingdutch’s picture

The (minor) risk I see with this is that people start adding diverging types to class methods that inherit a common, yet untyped method interface. Covariance would allow do so now, but then it will make life more complicated later once we will try to typehint the interface.

I think PHPStan can actually really help us here, in the way I described. I think that's something the Drupal Update Bot may be able to do even. However, it requires that we update coding guidelines that projects that consume Drupal should, before they upgrade a Drupal major version:

  • Ensure they're on Level 3 (or above) OR enable at least PHPStan\Rules\Methods\MethodSignatureRule
  • AND ensure the parameter reportMaybesInMethodSignatures: true is configured

If we consider changing the PHP return type of an interface as a breaking change and reserve it for majors (12.0.0), i.e.

interface Foo {
  // Before
  public function bar();
  // After, this would be a breaking change.
  public function bar() : string;
}

However, we ensure that we fix the PHPDocs correctly:

interface Foo {

  // Before
  /**
   * @return
   *   The return type.
   */
  public function bar();
  // After, this is not a breaking change and PHPStan starts flagging it downstream.
  /**
   * @return string
   *   The return type.
   */
  public function bar();
}

The above code would flag an implementation issue given the configuration described at the start: https://phpstan.org/r/f54ce223-907b-49b2-a45e-70e817347b13

catch’s picture

Status: Reviewed & tested by the community » Needs work

Needs another rebase.

bbrala’s picture

As per #26

Rebased and rebaselined. Will probably need to do again before commit, so here's the steps I took.

git reset --hard 96204beed3f
git rebase 11.x
composer install
vendor/bin/phpstan analyze --configuration=./core/phpstan.neon.dist --memory-limit=-1 --generate-baseline=core/.phpstan-baseline.php
git add core/.phpstan-baseline.php
git commit -m "Update baseline"
git push --force
smustgrave’s picture

Status: Needs work » Reviewed & tested by the community

Pushed! All I did was regenerate baseline.

bbrala’s picture

Status: Reviewed & tested by the community » Needs review

Back to NR after rebase.

Note; when you have done a composer install in core directory, things will fail ;p

And lol steven ;x

smustgrave’s picture

Don't want to step on toes can we revert the change to the README

longwave’s picture

Status: Needs review » Needs work

There is a change to README.md that is out of scope, seems your test commit for maintainer permissions ended up here ;)

bbrala’s picture

@smustgrave can you force push again?

I just noticed also, my 11.x was polluted by a small mini commit. I used the steps in the issue instead of my normal rebase workflow.

And that children, is why you always rebase on remote branches.

bbrala’s picture

*removed*

smustgrave’s picture

Status: Needs work » Reviewed & tested by the community

Done, think we are good now!

catch’s picture

Status: Reviewed & tested by the community » Fixed
Issue tags: -Needs documentation updates

Let's do it. Committed/pushed to 11.x, thanks!

Removing 'needs documentation updates' because this is already policy, the only new thing is the phpstan enforcement of it.

  • catch committed 4548e070 on 11.x
    Issue #3461318 by mstrelan, smustgrave, bbrala, catch, kingdutch,...

bbrala’s picture

Awesome!

joachim’s picture

This is causing problems when new code adds an existing trait, where the trait's methods don't have return types. e.g. #3390193: Add a drupalGet() method to KernelTestBase comment #38.

longwave’s picture

Similarly in #3452852: Add create() factory method with autowired parameters to PluginBase we are adding a new trait but can't add a return type because the trait is implemented in the base class and existing subclasses may already be overriding the method.

For these cases we just have to add to the baseline, I don't see a way around it.

mstrelan’s picture

We could add @phpstan-return void annotations to the methods in the traits (or an actual return type if it's safe to do so)

amateescu’s picture

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

xjm’s picture

Amending attribution.