Problem/Motivation

Quite often you want to only run a single method from a test.

Proposed resolution

Allow method names to be appended to the class name. For example if we want to run the testBookExport test in BookTest we pass the following class argument to run-tests.sh
--class 'Drupal\book\Tests\BookTest::testBookExport'

Remaining tasks

  • Review
  • Commit

User interface changes

None

API changes

None

Beta phase evaluation

Reference: https://www.drupal.org/core/beta-changes
Unfrozen changes Unfrozen because it only changes testing code

Comments

alexpott’s picture

Status: Active » Needs review
StatusFileSize
new2.12 KB

Let's stop commenting out code to run a single test method ftw.

dawehner’s picture

I really like that!

+++ b/core/scripts/run-tests.sh
@@ -625,10 +627,17 @@ function simpletest_script_run_one_test($test_id, $test_class) {
+    $test->run(array_filter([$method]));

This line is really confusing, it would be great to document that.

alexpott’s picture

StatusFileSize
new851 bytes
new2.19 KB

Simplified it a bit.

dawehner’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: +DX (Developer Experience), +Ghent DA sprint

That is great!

webchick’s picture

Status: Reviewed & tested by the community » Fixed

This seems like the kind of issue that'll only make it easier to get D8 done, so looks good to me. Unfortunately I don't believe it's possible to add automated tests for this.

Committed and pushed to 8.0.x. Thanks!

  • webchick committed 4378277 on 8.0.x
    Issue #2390749 by alexpott: run-tests.sh should allow single test...
David_Rothstein’s picture

Version: 8.0.x-dev » 7.x-dev
Status: Fixed » Patch (to be ported)
Issue tags: +Needs backport to D7

Seems like it could be backported, right?

  • webchick committed 4378277 on 8.1.x
    Issue #2390749 by alexpott: run-tests.sh should allow single test...
pietmarcus’s picture

Assigned: Unassigned » pietmarcus
Status: Patch (to be ported) » Needs review
StatusFileSize
new2.63 KB

Backported to Drupal 7. Also cleaned up a minor code issue: function simpletest_script_get_test_list was called twice unnecessarily. The results of the first call were not re-used. In the drupal-8 version of the script this was already fixed.

Please review!

Status: Needs review » Needs work

The last submitted patch, 9: 2390749-9.patch, failed testing.

pietmarcus’s picture

Status: Needs work » Needs review

Tests ran successfully this time. Please review!

David_Rothstein’s picture

The patch looks good and seems to work correctly.

The output it prints for "Tests to be run" is a little funny though (i.e. compared to running an entire class worth of tests):

$ php scripts/run-tests.sh --class UserAccountLinksUnitTests::testDisabledAccountLink,UserBlocksUnitTests

Drupal test run
---------------

Tests to be run:
 -  (UserAccountLinksUnitTests::testDisabledAccountLink)
 - User blocks (UserBlocksUnitTests)

Also the behavior is strange if you misspell the method name (especially compared to the existing behavior when you misspell the class name):

$ php scripts/run-tests.sh --class UserAccountLinksUnitTests::testDisabledAccountLinkWRONGNAME

Drupal test run
---------------

Tests to be run:
 -  (UserAccountLinksUnitTests::testDisabledAccountLinkWRONGNAME)

Test run started:
 Thursday, February 11, 2016 - 10:24

Test summary
------------

User account links 0 passes, 0 fails, and 0 exceptions

Test run duration: 0 sec

versus:

$ php scripts/run-tests.sh --class UserAccountLinksUnitTestsWRONGNAME::testDisabledAccountLink
  ERROR: Test class not found: UserAccountLinksUnitTestsWRONGNAME
  Did you mean?
  - UserAccountLinksUnitTests

These are pretty minor issues however - do we know if they are fixed in Drupal 8? (If not, leaving them for a followup would be fine.)

And one small comment about the code:

+        list($class_name, ) = explode('::', $test_class, 2);

Minor, but the comma isn't needed there - it could just be list($class_name).

Besides the above questions I think this should be RTBC.

berdir’s picture

Drupal 8 doesn't have test names, it always only shows the class name in run-tests.sh and also the UI.

Not sure if what we should do for 7.x.

pietmarcus’s picture

StatusFileSize
new1.52 KB
new3.6 KB

I have updated the patch with the remarks of #12. Information should be better now for end-users.

mikran’s picture

Status: Needs review » Needs work

--class TestCase::testMethod works but this patch breaks the original --class TestCase.

  • webchick committed 4378277 on 8.3.x
    Issue #2390749 by alexpott: run-tests.sh should allow single test...

  • webchick committed 4378277 on 8.3.x
    Issue #2390749 by alexpott: run-tests.sh should allow single test...
mikran’s picture

StatusFileSize
new3.19 KB

The patch didn't apply to latest core release. Rerolled.

  • webchick committed 4378277 on 8.4.x
    Issue #2390749 by alexpott: run-tests.sh should allow single test...

  • webchick committed 4378277 on 8.4.x
    Issue #2390749 by alexpott: run-tests.sh should allow single test...
jedihe’s picture

Status: Needs work » Needs review
StatusFileSize
new618 bytes
new3.2 KB

Fixed the bug causing --class TestClass to fail.

Tested scenarios:
- run-tests.sh ... MyTestGroup: detects all classes, and runs them
- run-tests.sh ... --class MyTestClass: runs only the test methods in the specified class
- run-tests.sh ... --class MyTestClass::testMethod: runs only the specified test method

jedihe’s picture

Had to manually cancel the tests, as this is not covered by tests.

jedihe’s picture

Assigned: pietmarcus » Unassigned
jedihe’s picture

Noticed that earlier patches were checked by the testbot, so decided to re-test #21. Tests are passing, so this should be ready for review.

mradcliffe’s picture

Hi @jedihe. We used to do back ports in the same issue, but we should create a follow-up issue for your patch in Drupal 7 and then mark this as Fixed and change the version back.

jedihe’s picture

Version: 7.x-dev » 8.0.x-dev
Issue summary: View changes
Status: Needs review » Fixed
Related issues: +#3209417: Backport from D8: run-tests.sh should allow single test methods to be run

Thanks @mradcliffe! I just filed the separate issue for the D7 backport. Marking this as fixed against 8.0.x-dev (latest 8.x I see used in this issue).

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.