Understanding Test Output

Test output on Drupal.org

Drupal.org directly parses the output of DrupalCI tests for display on Drupal.org. When tests fail, the results are displays as shown in the following screenshot.

See the results for the same test directly on the Jenkins dispatcher.

Comparison to the old PIFT/PIFR format

DrupalCI tests on Drupal.org provide test results output in an improved format compared to the prior PIFT/PIFR testing architecture.

PIFT/PIFR counted every assertion, so you would often see results like 100,000 tests passed, 15 failed.

DrupalCI counts the tests themselves, not individual assertions, so a result of 12000 passes 2 failures is more normal for drupal core.

Results are organized into Groups -> Classes -> Test Methods

Any non-pass is considered a failure, so there is no distinction between exceptions and failures.