This is the result of a test:

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite name="Drupal\sqlsrv\Tests\SqlServerSchemaTest" file="D:\d8\modules\sqlsrv\src\Tests\SqlServerSchemaTest.php" tests="7" assertions="50" failures="1" errors="0" time="75.154110">
    <testcase name="testPrimaryKeyHandling" class="Drupal\sqlsrv\Tests\SqlServerSchemaTest" file="D:\d8\modules\sqlsrv\src\Tests\SqlServerSchemaTest.php" line="31" assertions="5" time="2.041820"/>
    <testcase name="testPrimaryKeyAlter" class="Drupal\sqlsrv\Tests\SqlServerSchemaTest" file="D:\d8\modules\sqlsrv\src\Tests\SqlServerSchemaTest.php" line="73" assertions="1" time="58.366801"/>
    <testcase name="testUnsignedField" class="Drupal\sqlsrv\Tests\SqlServerSchemaTest" file="D:\d8\modules\sqlsrv\src\Tests\SqlServerSchemaTest.php" line="97" assertions="9" time="9.282572"/>
    <testcase name="testAddChangeWithIndex" class="Drupal\sqlsrv\Tests\SqlServerSchemaTest" file="D:\d8\modules\sqlsrv\src\Tests\SqlServerSchemaTest.php" line="207" assertions="4" time="1.903605"/>
    <testcase name="Warning" assertions="0" time="0.011641">
      <failure type="PHPUnit_Framework_AssertionFailedError">Warning
Test method "testAddChangeWithBinarySearchHelper" in test class "Drupal\sqlsrv\Tests\SqlServerSchemaTest" is not public.

</failure>
    </testcase>
    <testcase name="testAddChangeWithBinary" class="Drupal\sqlsrv\Tests\SqlServerSchemaTest" file="D:\d8\modules\sqlsrv\src\Tests\SqlServerSchemaTest.php" line="275" assertions="29" time="1.983175"/>
    <testcase name="testNumericFieldPrecision" class="Drupal\sqlsrv\Tests\SqlServerSchemaTest" file="D:\d8\modules\sqlsrv\src\Tests\SqlServerSchemaTest.php" line="350" assertions="2" time="1.564496"/>
  </testsuite>
</testsuites>

And this is what the ouput in the UI:

--

Simpletest UI is clearly missing out some important details regarding the PHPUnit warning.

It would also be more than interesting to be able to download the PHPUnit xml result file directly from the UI.

CommentFileSizeAuthor
#12 interdiff.txt1.06 KBMile23
#12 2608532_12.patch1.79 KBMile23
#10 2608532_10.patch2.62 KBMile23
#8 2608532_7.patch2.56 KBMile23
img.jpg339.96 KBdavid_garcia
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

david_garcia created an issue. See original summary.

david_garcia’s picture

david_garcia CreditAttribution: david_garcia commented
Issue summary: View changes
dawehner’s picture

dawehner
Primary language German
CreditAttribution: dawehner as a volunteer commented

IMHO we should do #2566767: Don't allow running phpunit-based tests via the UI

david_garcia’s picture

david_garcia CreditAttribution: david_garcia commented
Related issues: +#2566767: Don't allow running phpunit-based tests via the UI
Mile23’s picture

Mile23
Primary language English
Location Seattle, WA
CreditAttribution: Mile23 as a volunteer commented
Title: Running PHPUnit tests through the UI is hidding details when warnings are triggered and there is no way to download the PHPUnit results. » Simpletest UI munges PHPUnit-based test output

This is completely fixable.

Whether it's worth it to fix to anyone is another story.

A lot of core devs are using CLI tools to run tests, but we don't necessarily want that to normalize everything for everyone else.

Mile23’s picture

Mile23
Primary language English
Location Seattle, WA
CreditAttribution: Mile23 as a volunteer commented

First step: #2641632: Refactor simpletest's *_phpunit_*() (and junit) functions etc. to a class, deprecate

Mile23’s picture

Mile23
Primary language English
Location Seattle, WA
CreditAttribution: Mile23 as a volunteer commented

Discovered while assessing: #2661542: Isolate Drupal\Tests\Component\Plugin\DefaultFactoryTest from core test module

Mile23’s picture

Mile23
Primary language English
Location Seattle, WA
CreditAttribution: Mile23 as a volunteer commented
Status: Active » Needs review
FileSize
2608532_7.patch2.56 KB

So basically if you generate an error, the test ID is forced to be 1.

This means that no test results end up in the query when it comes time to show you the error.

There are also sort of two 'modes': Running one unit test and running more than one unit test. If you run more than one test, you might see #2661542: Isolate Drupal\Tests\Component\Plugin\DefaultFactoryTest from core test module

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Mile23’s picture

Mile23
Primary language English
Location Seattle, WA
CreditAttribution: Mile23 as a volunteer commented
FileSize
2608532_10.patch2.62 KB

Reroll.

dawehner’s picture

dawehner
Primary language German
CreditAttribution: dawehner as a volunteer and at Chapter Three commented

Nice patch!

+++ b/core/modules/simpletest/simpletest.module
@@ -269,6 +271,7 @@ function simpletest_phpunit_run_command(array $unescaped_test_classnames, $phpun
   $command = array(
     $phpunit_bin,
+    '-c ' . \Drupal::root() . '/core',
     '--log-junit',
     escapeshellarg($phpunit_file),
   );
@@ -291,16 +294,10 @@ function simpletest_phpunit_run_command(array $unescaped_test_classnames, $phpun

@@ -291,16 +294,10 @@ function simpletest_phpunit_run_command(array $unescaped_test_classnames, $phpun
     ));
   }
 
-  // Need to change directories before running the command so that we can use
-  // relative paths in the configuration file's exclusions.
-  $old_cwd = getcwd();
-  chdir(\Drupal::root() . "/core");
-
   // exec in a subshell so that the environment is isolated when running tests
   // via the simpletest UI.
-  $ret = exec(join($command, " "), $output, $status);
+  $ret = exec(join($command, ' '), $output, $status);
 
-  chdir($old_cwd);
   putenv('SIMPLETEST_DB=');

Nice! I'm wondering though whether this change is needed to fix this issue.

Mile23’s picture

Mile23
Primary language English
Location Seattle, WA
CreditAttribution: Mile23 as a volunteer commented
FileSize
2608532_12.patch1.79 KB
interdiff.txt1.06 KB

Yah, not so necessary.

Removed.

There still needs to be some follow-up on the simpletest ui form, because wow is it ever messed up. But we'll need the work here to be committed before starting on that.

Status: Needs review » Needs work

The last submitted patch, 12: 2608532_12.patch, failed testing.

The last submitted patch, 12: 2608532_12.patch, failed testing.

Mile23’s picture

Mile23
Primary language English
Location Seattle, WA
CreditAttribution: Mile23 as a volunteer commented
Status: Needs work » Needs review

Unrelated migration test fail. I wonder when someone will finally fix that. :-)

Re-running test.

Status: Needs review » Needs work

The last submitted patch, 12: 2608532_12.patch, failed testing.

Mile23’s picture

Mile23
Primary language English
Location Seattle, WA
CreditAttribution: Mile23 as a volunteer commented
Status: Needs work » Needs review
Mile23’s picture

Mile23
Primary language English
Location Seattle, WA
CreditAttribution: Mile23 as a volunteer commented

This might be closed as duplicate of #2575725: Run PHPunit tests one at a time via UI to avoid exception when selecting too many

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.6 was released on August 2, 2017 and is the final full bugfix release for the Drupal 8.3.x series. Drupal 8.3.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.4.0 on October 4, 2017. (Drupal 8.4.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.4.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.4 was released on January 3, 2018 and is the final full bugfix release for the Drupal 8.4.x series. Drupal 8.4.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.5.0 on March 7, 2018. (Drupal 8.5.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.5.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.6 was released on August 1, 2018 and is the final bugfix release for the Drupal 8.5.x series. Drupal 8.5.x will not receive any further development aside from security fixes. Sites should prepare to update to 8.6.0 on September 5, 2018. (Drupal 8.6.0-rc1 is available for testing.)

Bug reports should be targeted against the 8.6.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.8.x-dev

Drupal 8.6.x will not receive any further development aside from security fixes. Bug reports should be targeted against the 8.8.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.9.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.7 was released on June 3, 2020 and is the final full bugfix release for the Drupal 8.8.x series. Drupal 8.8.x will not receive any further development aside from security fixes. Sites should prepare to update to Drupal 8.9.0 or Drupal 9.0.0 for ongoing support.

Bug reports should be targeted against the 8.9.x-dev branch from now on, and new development or disruptive changes should be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

quietone’s picture

quietone CreditAttribution: quietone as a volunteer commented
Project: Drupal core » SimpleTest
Version: 8.9.x-dev » 8.x-3.x-dev
Component: simpletest.module » Code

Triaging issues in simpletest.module as part of the Bug Smash Initiative to determine if they should be in the Simpletest Project or core.

This looks like it belongs in the Simpletest project.