Purpose
-------
Update XML output to support CircleCI (move closer to the output of PHPUnit).

Reproducible Steps
----------------
php ./scripts/run-tests.sh --concurrency 4 --xml "/var/www/sitename/xmloutputdirectory/" --all
- or the faster version -
php ./scripts/run-tests.sh --concurrency 4 --xml "/var/www/sitename/xmloutputdirectory/" SpecifcGroupNameHere

Example Output (broken)
---------------------

<testsuite><testcase classname="ModuleNameExampleTestUnitTestCase" name="testExample()"><failure type="failure" message="Other">This is just a test.</failure></testcase></testsuite>

Expected Behavior (output)
-----------------------

<testsuite><testcase classname="ModuleNameExampleTestUnitTestCase" name="testExample()"><failure type="failure" message="This is just a test."></failure></testcase></testsuite>

NOTICE: That the message attribute contains the desired message now.

TIP: This works with CircleCI

CommentFileSizeAuthor
#3 simpletest-junit-2883175-3-D8.patch1.29 KBccjjmartin
#2 simpletest-junit-2883175-2-D7.patch1.27 KBccjjmartin

Comments

ccjjmartin created an issue. See original summary.

ccjjmartin’s picture

ccjjmartin commented
Status: Needs work » Needs review
StatusFileSize
new simpletest-junit-2883175-2-D7.patch1.27 KB
ccjjmartin’s picture

ccjjmartin commented
Version: 7.x-dev » 8.4.x-dev
StatusFileSize
new simpletest-junit-2883175-3-D8.patch1.29 KB

I guess this should be done in 8.4.x first (adding patch).

ccjjmartin’s picture

ccjjmartin commented

NOTICE to testers. I have not tested the D8 version of this yet. I also have not tested an exception yet. I can verify this works on D7 for normal failures, example test code for a "normal" failure:


/**
 * @file
 * A simple test to verify that this module is functioning properly.
 */

/**
 * Basic test to check the test failures are functioning properly.
 */
class ModuleNameExampleUnitTestCase extends DrupalUnitTestCase {

  /**
   * Give display information to the SimpleTest system.
   *
   * GetInfo() returns a keyed array of information for SimpleTest to show.
   *
   * It's a good idea to organize your tests consistently using the 'group'
   * key.
   */
  public static function getInfo() {
    return array(
      'name' => 'Example Test',
      'description' => 'Ensure that test failures are functioning properly.',
      'group' => 'Example',
    );
  }

  /**
   * A simple test to verify tests are failing properly.
   */
  public function testExample() {
    $message = 'This is just a test.';
    $this->fail($message);
  }

}
ccjjmartin’s picture

ccjjmartin commented
Title: Simpletest XML output formatting change » CircleCI Simpletest XML output formatting change
Category: Bug report » Feature request
Priority: Normal » Minor
Issue summary: View changes
Status: Needs review » Needs work

Some further information on this change:

1) Per discussion with a core maintainer this needs tests.

2) Some further research (Googling) on JUnit testing output shows that there isn't a set standard (so if we change what we have we are probably going to break someones workflow). This could be implemented as a separate xml formatting output to not break the existing flow.

3) CircleCI shows that they are using the expected output of PHPUnit which is why the existing format is broken: https://circleci.com/docs/1.0/test-metadata/#phpunit so this change to be complete should use a format closer to the output of this code: https://github.com/sebastianbergmann/phpunit/blob/master/src/Util/Log/JU...

Seems that because of this, this is more of a feature request to support CircleCI.

mile23’s picture

mile23
Primary language English
Location Seattle, WA
commented
Issue tags: +run-tests.sh
Related issues: +#2834033: Add a junit format to run-tests.sh

Related, maybe duplicate: #2834033: Add a junit format to run-tests.sh

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

smustgrave’s picture

smustgrave commented
Status: Needs work » Postponed (maintainer needs more info)
Issue tags: +stale-issue-cleanup

Thank you for sharing your idea for improving Drupal.

We are working to decide if this proposal meets the Criteria for evaluating proposed changes. There hasn't been any discussion here for over 8 years which suggests that this has either been implemented or there is no community support. Your thoughts on this will allow a decision to be made.

Since we need more information to move forward with this issue, the status is now Postponed (maintainer needs more info). If we don't receive additional information to help with the issue, it may be closed after three months.

Thanks!

longwave’s picture

longwave
he/him
Primary language English
Location UK
commented
Status: Postponed (maintainer needs more info) » Closed (won't fix)

Only one person requested this, over eight years ago. This is a niche feature that I don't think we need to support in core given the lack of interest, therefore closing this one out.

Now that this issue is closed, please review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, please credit people who helped resolve this issue.