Problem/Motivation
Implementing strict typing in existing code is challenging. If we don't enforce it on new code we'll constantly be chasing our tail. We get missing type checks out of the box with phpstan level 6, but we're currently stuck on level 1.
This issue proposes enabling just the rules required to enforce return types, while remaining at the current level.
Steps to reproduce
N/A
Proposed resolution
- Enable
MissingFunctionReturnTypehintRuleandMissingMethodReturnTypehintRulephpstan rules - Ignore
missingType.iterableValue - Update the phpstan baseline to include all violations of this rule
Remaining tasks
Review the changes to phpstan.neon.dist and what's currently being added to the baseline, even if it's not mergeable, to verify that we're only adding violations of identifier: missingType.return.
For example:
# Check how many new errors are ignored.
$ git diff 11.x | grep '^\+$ignoreErrors\[]' | wc -l
12378
# Check how many 'missingType.return' errors are added.
$ git diff 11.x | grep '+.*// identifier:' | grep 'missingType.return' | wc -l
12378
# Check what other lines are removed.
$ git diff 11.x | grep '^\-\s'
- // identifier: variable.undefined
- 'message' => '#^Variable \\$patterns might not be defined\\.$#',
- 'message' => '#^Missing cache backend declaration for performance\\.$#',
- // identifier: variable.undefined
- 'message' => '#^Variable \\$inner_count might not be defined\\.$#',
Baseline size comparison
Before: 2640 lines, 115K
After: 76908 lines, 3.7M
User interface changes
API changes
Data model changes
Release notes snippet
All new functions and methods must have return types defined.
| Comment | File | Size | Author |
|---|---|---|---|
| #7 | 3461318-nr-bot.txt | 90 bytes | needs-review-queue-bot |
Issue fork drupal-3461318
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
- 3461318-enforce-return-types
changes, plain diff MR !8763
Comments
Comment #3
mstrelan commentedComment #4
andypostStats for baseline looks big but looks like it should be better then growing amount of newcode
Comment #5
smustgrave commentedMR appears to be unmergable
Comment #6
mstrelan commentedObviously it will need constant rebasing. Setting back to NR to get actual review.
Comment #7
needs-review-queue-bot commentedThe Needs Review Queue Bot tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".
This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.
Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.
Comment #8
mstrelan commentedLet's just review the idea and the changes to
phpstan.neon.dist. If we're happy with that then we can regenerate the baseline. We can also review what's currently being added to the baseline, even if it's not mergeable, to verify that we're onlyidentifier: missingType.return.Comment #9
smustgrave commentedEnable MissingFunctionReturnTypehintRule and MissingMethodReturnTypehintRule phpstan rules
What if we broke these out into 2 tickets would that be easier?
Comment #10
mstrelan commentedNot sure there's any difference reviewing 10000 or 17000 changes.
Comment #11
smustgrave commentedWould it be easier to breakout into groups of components or group of modules?
Just brainstorming to help keep the idea moving.
Comment #12
mstrelan commentedI appreciate that the MR is massive but really the only change that needs review at all is to phpstan.neon.dist. Everything else is proven by the phpstan job passing. I don't think committing in chunks will help, this probably just needs "Needs ____ review" but I don't know what ____ is, maybe FM?
Comment #13
smustgrave commentedFramework would probably be best. And would just ping them.
Comment #14
quietone commentedCan you add the current size of the baseline and the size with this MR?
Comment #15
mstrelan commentedComment #16
mstrelan commentedRebased, re-baselined and rewrote the issue summary. FWIW I don't know why some lines were removed from the baseline, but that can only be a good thing as long as the phpstan job still passes.
Comment #17
smustgrave commentedSo what would be the scope or game plan to address these eventually? So people aren’t doing a bunch of tickets
Comment #18
mstrelan commentedThis issue is not about adding the return types to existing methods, it's just about preventing new methods from being added that don't have a return type. I think adding return types to methods in run time code needs BC consideration most of the time.
Comment #19
acbramley commentedBig +1 on this and agree with #12. The longer we wait to do this the larger the potential baseline as more and more code gets added, I know we're pretty good at asking for return types but this would enforce it.
Comment #20
spokje*jumps on the +1 bandwagon*
Having tried to work on PHPStan lvl 1 issues in the current baseline in a previous life, I very much think this is the case.
I see a slight risk of people adding a new
missingType.returnerror to the baseline, since they're seeing the current ones also being in the baseline, but with a explanatory CR and, since almost nobody will read that, a vigilant NR and core committers crew (so no problem there :) we will be fine.Comment #21
smustgrave commentedLet’s see if it can get in
Comment #22
smustgrave commentedAlso imagine if this gets in a number of tickets may get sent back
Comment #23
bbralaThis is a great step imo to start enforcing.
This might also need an update to the Coding Standards. And if this is enforced do we need to say something about new classes that might not be able to add since they for example implement an interface or extend a base class? There might be place where this is imossoble.
Comment #24
mstrelan commentedDue to covariance you can add a return type to a child class when the parent or interface doesn't have one. See https://www.php.net/manual/en/language.oop5.variance.php
Comment #25
catchThis needs another re-roll, I'm +1. Pinged the other committers to try to get sign-offs.
Comment #26
mstrelan commentedRebased and rebaselined. Will probably need to do again before commit, so here's the steps I took.
Comment #30
catchDiscussed this briefly with @xjm, @longwave, @larowlan and @quietone - we're all +1 so I am removing the RM/FM tags.
@xjm suggested doing a core blog post announcing the change since potentially a lot of existing MRs in the queue would start failing on PHPStan (even if they hopefully would have been eventually marked needs work via manual review). I'm not sure this is necessary because this is already a coding standard, we're just adding it to phpstan.
Double checked and this is already clearly documented in the coding standards exactly how we'll be enforcing it in PHPStan https://www.drupal.org/docs/develop/standards/php/php-coding-standards#s...
Comment #31
andypostIt needs docs update and blog post is a good idea as well
I think it could be done after commit and will bring nice amount of rerolls for Barcelona's sprints
Comment #32
kingdutchAlthough this does go against the PHPStan philosophy which is "Don't pick and choose which rules you apply, it works best if you apply all of them", at this point I'm happy to have any progress towards better type support in Drupal.
The only thing I'd want to add is that this doesn't change the need for the related issues mentioned in the parent. Even though it provides an important step in the right direction by adding return type hints for new code.
Also unrelated to the issue at hand but important to share with everyone in this thread vis-a-vis "how to add return types to interfaces": PHPStan is perfectly happy accepting a return type as PHPDoc as per the following example.
This means that as long as we force the rule to be enabled and downstream projects enable it then a viable strategy is:
1. Ensure all interfaces have a properly typed `@return` type annotation
2. In a major version promote the return type from PHPDoc to PHPCode.
PHPStan will already start reporting the return type mis-match based on the PHPDoc in downstream projects and help them implement it on classes (because as mentioned earlier in this thread, a class may always return a narrower type than what it extends/implements thanks to covariance). That means that as long as they fix those PHPStan errors before making the major jump, the major version jump won't actually be breaking for them. This aligns well with our current "Fix all deprecation on the last minor and then jump to the next major without changes" philosophy.
Comment #33
mondrakeThe (minor) risk I see with this is that people start adding diverging types to class methods that inherit a common, yet untyped method interface. Covariance would allow do so now, but then it will make life more complicated later once we will try to typehint the interface.
See #3470913: Ensure getWidth()/getHeigth always return ?int and Symfony's upstream [ErrorHandler] Add support for @return-type-will-change #58134 for a different (but certainly longer) approach.
Comment #34
kingdutchI think PHPStan can actually really help us here, in the way I described. I think that's something the Drupal Update Bot may be able to do even. However, it requires that we update coding guidelines that projects that consume Drupal should, before they upgrade a Drupal major version:
PHPStan\Rules\Methods\MethodSignatureRulereportMaybesInMethodSignatures: trueis configuredIf we consider changing the PHP return type of an interface as a breaking change and reserve it for majors (12.0.0), i.e.
However, we ensure that we fix the PHPDocs correctly:
The above code would flag an implementation issue given the configuration described at the start: https://phpstan.org/r/f54ce223-907b-49b2-a45e-70e817347b13
Comment #35
catchNeeds another rebase.
Comment #36
bbralaAs per #26
Rebased and rebaselined. Will probably need to do again before commit, so here's the steps I took.
Comment #37
smustgrave commentedPushed! All I did was regenerate baseline.
Comment #38
bbralaBack to NR after rebase.
Note; when you have done a composer install in core directory, things will fail ;p
And lol steven ;x
Comment #39
smustgrave commentedDon't want to step on toes can we revert the change to the README
Comment #40
longwaveThere is a change to README.md that is out of scope, seems your test commit for maintainer permissions ended up here ;)
Comment #41
bbrala@smustgrave can you force push again?
I just noticed also, my 11.x was polluted by a small mini commit. I used the steps in the issue instead of my normal rebase workflow.
And that children, is why you always rebase on remote branches.
Comment #42
bbrala*removed*
Comment #43
smustgrave commentedDone, think we are good now!
Comment #44
catchLet's do it. Committed/pushed to 11.x, thanks!
Removing 'needs documentation updates' because this is already policy, the only new thing is the phpstan enforcement of it.
Comment #47
bbralaAwesome!
Comment #48
joachim commentedThis is causing problems when new code adds an existing trait, where the trait's methods don't have return types. e.g. #3390193: Add a drupalGet() method to KernelTestBase comment #38.
Comment #49
longwaveSimilarly in #3452852: Add create() factory method with autowired parameters to PluginBase we are adding a new trait but can't add a return type because the trait is implemented in the base class and existing subclasses may already be overriding the method.
For these cases we just have to add to the baseline, I don't see a way around it.
Comment #50
mstrelan commentedWe could add
@phpstan-return voidannotations to the methods in the traits (or an actual return type if it's safe to do so)Comment #51
amateescu commentedAnother instance of the problem mentioned above: https://git.drupalcode.org/issue/drupal-3425081/-/jobs/3144156 (from #3425081: Integrate Navigation with Workspaces )
Comment #53
xjmAmending attribution.