Looking over the unit tests for Drupal\Component\Utilities I noticed some of the documentation could use some cleanups. Mostly converting @see to @covers.

Couple other things:

  1. There was also a lot of code missing tests so I wrote a couple and even found a bug in UrlHelper::externalIsLocal that made it impossible to match urls unless there was no path given.
  2. UserAgent didn't have any unit tests. It did have what amounted to a UnitTest in a WebTests in language though so I converted it.
  3. Number had two test files that didn't need to be split.

Beta phase evaluation

Reference: https://www.drupal.org/core/beta-changes
Issue category Task
Unfrozen changes Unfrozen because it only changes internal unit tests
Prioritized changes The main goal of this issue is cleanup of unit tests.

Comments

Status: Needs review » Needs work

The last submitted patch, UtilitiesTestCleanup.patch, failed testing.

mile23’s picture

Potentially overlapping issue here: #2298667: Code review for report generation

anavarre’s picture

Coming here from #2076325: Drupal\Component\Utility\Url::externalIsLocal() is wrong in common cases. I wonder if this is the source of the below error that we're trying to address in this issue?

$ phpunit -c core/phpunit.xml.dist --coverage-html /tmp/report
PHPUnit 4.1.3 by Sebastian Bergmann.

Configuration read from /var/www/html/head/core/phpunit.xml.dist

..............................................   61 / 5258 (  1%)
...............................................  122 / 5258 (  2%)
...............................................  183 / 5258 (  3%)
...............................................  244 / 5258 (  4%)
...............................................  305 / 5258 (  5%)
...............................................  366 / 5258 (  6%)
...............................................  427 / 5258 (  8%)
.......Trying to @cover or @use not existing method "\Drupal\Component\Utility\Url::externalIsLocal".

FYI, 2 days ago it was not failing that early (but failing, still). I don't see anything else in the commits but #2076325: Drupal\Component\Utility\Url::externalIsLocal() is wrong in common cases. that would have changed anything.

mile23’s picture

Yah, it looks like a bad @coversDefaultClass made it through #2076325: Drupal\Component\Utility\Url::externalIsLocal() is wrong in common cases., but the code and the test are good.

The patch at the top no longer applies.

neclimdul’s picture

StatusFileSize
new49.79 KB

Not sure how I let that through on that other issue, sorry. Re-roll and some minor doc fixes. Had to rebase on the getInfo() removal so no clean interdiff.

neclimdul’s picture

Status: Needs work » Needs review

testbot and review

mile23’s picture

  1. +++ b/core/tests/Drupal/Tests/Component/Utility/SortArrayTest.php
    @@ -20,6 +23,8 @@ class SortArrayTest extends UnitTestCase {
        * @dataProvider providerSortByWeightElement
    +   * @covers ::sortByWeightElement
    +   * @covers ::sortByKeyInt
    
    @@ -98,6 +99,8 @@ public function providerSortByWeightElement() {
        * @dataProvider providerSortByWeightProperty
    +   * @covers ::sortByWeightProperty
    +   * @covers ::sortByKeyInt
    
    @@ -176,6 +175,8 @@ public function providerSortByWeightProperty() {
        * @dataProvider providerSortByTitleElement
    +   * @covers ::sortByTitleElement
    +   * @covers ::sortByKeyString
    
    @@ -247,6 +244,8 @@ public function providerSortByTitleElement() {
        * @dataProvider providerSortByTitleProperty
    +   * @covers ::sortByTitleProperty
    +   * @covers ::sortByKeyString
    
    @@ -318,6 +313,9 @@ public function providerSortByTitleProperty() {
        * @dataProvider providerTestSortByWeightAndTitleKey
    +   * @covers ::sortByWeightAndTitleKey
    +   * @covers ::sortByKeyString
    +   * @covers ::sortByKeyInt
    

    It looks like these tests are double- and triple-dipping for coverage. The test only directly calls the first one in all of those cases, and there aren't any tests for some of the other ones.

  2. +++ b/core/tests/Drupal/Tests/Component/Utility/TimerTest.php
    @@ -11,15 +11,20 @@
    -   * @see \Drupal\Component\Utility\Timer::read()
    +   * @covers ::start
    +   * @covers ::stop
    +   * @covers ::read
    

    This is a functional test so it doesn't cover anything. Should be marked @coversNothing.

  3. +++ b/core/tests/Drupal/Tests/Component/Utility/UnicodeTest.php
    @@ -11,20 +11,30 @@
    +  /**
    +   * {@inheritdoc}
    +   *
    +   * @covers ::check
    +   */
       public function setUp() {
         // Initialize unicode component.
         Unicode::check();
       }
    

    Can a setUp() really be considered to cover some code?

Some other @covers beyond that.

neclimdul’s picture

1) Multiple coverage of internal methods as its called different ways.
2) This is the only testing we do of those methods and since they're all connected, it is coverage as far as I'm concerned. Breaking up coverage would be pretty tedious as well.
3) Yes, I've been told this is correct in other issues.

sun’s picture

I don't understand why this patch performs the following change everywhere:

   /**
    * Tests external versus internal paths.
    *
+   * @dataProvider providerTestIsExternal
+   * @covers ::isExternal
+   *
    * @param string $path
...
-   *
-   * @dataProvider providerTestIsExternal
-   * @covers ::isExternal
    */
   public function testIsExternal($path, $expected) {

Can we keep the functional method annotations closer to the method definition, please? (i.e., leave them were they are)

Especially a specified @dataProvider should be as quickly visible as possible. By moving it up, you first have to scroll away that other, overly verbose PHPDoc garbage.

anavarre’s picture

Just to note that today I'm seeing another failure, a bit later in the tests.

$ phpunit -c core/phpunit.xml.dist --coverage-html /tmp/report
PHPUnit 4.1.3 by Sebastian Bergmann.

Configuration read from /var/www/html/head/core/phpunit.xml.dist

.............................................  427 / 5323 (  8%)
.............................................  488 / 5323 (  9%)
.............................................  549 / 5323 ( 10%)
.............................................  610 / 5323 ( 11%)
.............................................  671 / 5323 ( 12%)
.....Trying to @cover or @use not existing method "\Drupal\Core\Asset\LibraryDiscoveryParser::buildLibrariesByExtension".

Looks related to #2276219: Asset libraries should declare their license

mile23’s picture

@anvarre: That's a bit out of scope for this issue, but it's true that if we're using @covers (and we are), then the @covers annotation should at least be accurate. :-) Also, generating a coverage report should be part of the continuous integration process so these errors are caught up-front.

neclimdul’s picture

Reroll.

Can we keep the functional method annotations closer to the method definition, please? (i.e., leave them were they are)

It seems the @params and @return types would be better closer. They explain what the arguments to the function are, which in the case of dataproviders is actually more important to understanding the test then reading what function is providing it. We where all over in placement even within individual test files so as I was fixing some, I moved others to match.

The _actual_ reason I went with this other then skimming and it seeming to be the more common option, was the short description, @group, @covers, and @dataprovider all describe the test metadata and the short description is forced to the top and @group tends to be at the top so I put all this at the top.

We should probably discuss doc standards in a doc standards issue though if this needs to be a standard. And if this absolutely blocks this issue I'll undo the changes but obviously I'd rather just get it in rather then spending the time on it.

sun’s picture

AFAIK, that issue is #2253915: [policy, no patch] Fix coding standards for PHPUnit tests

As also noted over there, I strongly disagree with applying our overly verbose PHPDoc standards to PHPUnit tests/methods. IMO, @param and @return shouldn't even exist; all unit test code should (or actually MUST) be of trivial complexity; if you need to explain anything, then your test has a fundamental problem. The vast majority of unit test methods should not have any PHPDoc at all; most of the rest should be happy with a single @dataProvider annotation, nothing else.

Your thinking of standardization/harmonization is very appreciated (because quality), but yeah, I think you've picked and applied/adopted the wrong baseline here. It would be great if we can undo those changes and defer the standards discussion to #2253915.

mile23’s picture

StatusFileSize
new48.46 KB

Re-roll.

Status: Needs review » Needs work

The last submitted patch, 14: 2294503_14.patch, failed testing.

mile23’s picture

Status: Needs work » Needs review
StatusFileSize
new48.08 KB
new3.26 KB

And this should work.

The main problem was UserAgentTest, which contained a reference to \Drupal\Core\Language\Language, and the deprecated randomName().

I refactored the setUp() method into two methods which supply language codes and mappings. Using $this->someProperty for dependencies in unit tests is generally bad form.

mile23’s picture

@neclimdul: I'd split this out into two more issues, one for UserAgentTest and one for HtmlTest, leaving the actual cleanup part here.

I'd do it but it's really your code. :-)

mile23’s picture

jhedstrom’s picture

Issue summary: View changes

Ideally this would have been separate issues, but I don't think that's crucial.

Added a beta evaluation.

jhedstrom queued 16: 2294503_16.patch for re-testing.

mile23’s picture

StatusFileSize
new34.61 KB
new13.47 KB

Here's the cleanup without the extra tests (UserAgentTest and HtmlTest) or changes to phpunit.xml.dist.

Note that I removed the UserAgent test and put it back into LanguageBrowserDetectionUnitTest.

Followups to follow. :-)

mile23’s picture

jhedstrom’s picture

Status: Needs review » Reviewed & tested by the community

#21 is good work towards standardizing how the phpunit tests are written, and is allowed under beta evaluation.

alexpott’s picture

Status: Reviewed & tested by the community » Fixed

Committed 65f1d49 and pushed to 8.0.x. Thanks!

Thanks for adding the beta evaluation for to the issue summary.

  • alexpott committed 65f1d49 on 8.0.x
    Issue #2294503 by Mile23, neclimdul: Component Utilities unit test...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.