Problem/Motivation
There are several things phpUnit can view as "Risky". Generally these are things that are prone to failure, false positives, and/or false negatives. This patch fixes tests that fall into 2 of those categories.
1) Tests that don't assert anything
2) Tests that change global state.
Most of the missing assertions where tests that called ->fail directly on a failure condition instead of asserting the condition.
Most of the global state tests I just cleaned up after since they actually used them.
Pretty straight forward.
Beta phase evaluation
| Issue category | Task because its just hardening/cleaning up tests |
|---|---|
| Issue priority | Minor, tests should all probably work and continue working without these changes. |
| Unfrozen changes | Tests |
| Disruption | Should be none. Minor changes to tests and tests that are unlikely to be touched by other issues. |
| Comment | File | Size | Author |
|---|---|---|---|
| #9 | 2542486-9.interdiff.txt | 699 bytes | neclimdul |
| #9 | fix_risky_phpunit_tests-2542486-9.patch | 8.48 KB | neclimdul |
| #8 | interdiff.txt | 1.32 KB | wim leers |
| #8 | fix_risky_phpunit_tests-2542486-8.patch | 7.8 KB | wim leers |
Comments
Comment #1
dawehnerYeah, we don't have any risky tests :)
Comment #2
jibranROFL
Comment #3
neclimdullol, cone of shame
Comment #4
mile23:-)
+1 for the concept. Don't have time for a proper review ATM.
Comment #5
mile23It looks like the way to repro this is to:
cd core;./vendor/bin/phpunit -v. (Include-vto get PHPUnit to tell you which tests are risky.)Which is what happens, so yay, mission accomplished.
Some review stuff:
Not really a test of much, is it? You've swapped one kind of risky test for another. :-)
Rather than
assertTrue(TRUE), we should ignore$this->specialAttributesRouteSubscriberand mock our own. That way we can set expectations for all the stuffSpecialAttributesRouteSubscriber::onAlterRoutes()does.currentUserRole is a string... Why unset it? It's about to go out of scope anyway.
Comment #6
neclimdul1) It sort of does something because the phpunit error handler throws and exception but yeah its pretty weak. #2364467: SpecialAttributesRouteSubscriber::onAlterRoutes() doesn't return a value does it better so it just took it out.
2) I don't know what that's from but I re-factored that code several times so its probably useless tinkering. I can't see a point.
Comment #7
mile23#2364467: SpecialAttributesRouteSubscriber::onAlterRoutes() doesn't return a value. Nice. An issue I'd forgotten I made, lost to the ravages of antiquity, reborn to new life though the use of the search function!
This looks good, and we can then argue about mocking in that other issue. :-)
Comment #8
wim leersFixed my remarks so this can stay at RTBC:
These changes don't belong here. That's not what the test is testing. Should be reverted.
Missing docs.
Comment #9
neclimdulAppreciate the effort but that adds back in a the risky test we where fixing :(.
This seems more correct assertion though.
Comment #10
wim leers@neclimdul: huh? That test only is the inverse of
RendererPlaceholdersTest::testInvalidLazyBuilderArguments(): it asserts that no exceptions are thrown when only scalar arguments are specified for the#lazy_builder. It should not assert any of the output.Where the risk there? There's no global involved.
Comment #11
mile23The risky part is that there are no assertions in that test. There is no expected behavior, since we can't specify that no exception was thrown.
One solution is to decide on some behavior to test, and having it match the documented return type and value is reasonable and it gives us information in the future if things change.
Another way to do it would be a try/catch block with the catch failing. This more explicitly says what you're testing, so it might be better.
Or, you know.. Combine the two. :-)
This patch fixes all the risky tests except for
SpecialAttributesRouteSubscriberTest::testOnRouteBuildingValidVariables(), which is marked as risky after applying the patch at https://www.drupal.org/node/2105583#comment-10176962Comment #12
neclimdulYeah from IS
Generally I think of this as needing a basic assertion to say "the code ran." Ideally "does not throw an exception" tests should be _very_ specific though and other unit tests will cover this 99% of the time.
I'm not sure from the last two comments what work needs to be done. If we need to separate this test from the other cleanups I'm ok with making another issue to clean it up like we did with the Subscriber though.
Comment #13
wim leersAhhh! That makes sense. Thanks :) Sorry for missing that!
Comment #14
alexpottTest only change. Committed 4710b09 and pushed to 8.0.x. Thanks!