Sometimes the output of tests that you get in your browser can be cluttered between other blocks or because some tests output very long messages.

Also if you want to run successively the same tests in different conditions, even different platforms, it could be nice to be able to compare the results programmatically.

You may want to present the results of the tests differently to include in your own reports or documents.

XML would be lovely for that, since it leaves the choice between many tools and languages to process it.
Or, as I often do, simply read it as you would read a textual report, provided that it is compact enough
(using attributes where suitable, avoiding meaningless nesting ...).

As an alternative, we could use the php var_export() to output php source, which could be processed externally.

What do you think ?

If I knew simpletest code a little better, I could try to give a hand on this task.
That's why I started by generating simpletest api documentation.

Comments

boombatower’s picture

boombatower commented
Title: Generate XML reports » SimpleTest: abstract reporter and generate XML reports
Project: SimpleTest » Drupal core
Version: 6.x-2.x-dev » 7.x-dev
Component: User interface » simpletest.module

This needs to be implement in D7 and backported.

jazzslider’s picture

jazzslider commented

Hello!

I'd made some comments about this same issue here: #495246: Output options for command-line test runner What I noticed was that the original SimpleTest library has a variety of flexible reporter classes that can be swapped out as necessary; I'd like to see something like that implemented in the Drupal version, since XML output would really help in certain build automation processes (I rarely remember to run my tests manually :)

I realize that the current implementation of Drupal simpletest is pretty dramatically different from its namesake library in this regard, and insofar as folks are only running their tests from the web interface I don't suppose that's a problem…but the command-line test runner really needs this extra flexibility.

If someone more familiar with Drupal simpletest than I am could identify the specific parts of the code responsible for test results output in both contexts (web interface and CLI), I'd be happy to look a little closer. For the web interface, I'm guessing it's just the two theme_* functions? In any case, what is needed here is a set of reusable reporter classes around which the existing code can wrap.

Unfortunately I have absolutely no experience with Drupal 7 (the module I'm working on runs on D6), so I may not be the best person to attempt this there…but I'd be happy to offer what suggestions and/or code I can!

boombatower’s picture

boombatower commented
Status: Active » Postponed

That is the problem right now...there isn't a unfied test-runner/reporter...I plan to work on that, but until then I would say we postpone this.

The results are store in the db...so you can extract them that way if you need to...that is what testing.drupal.org does.

ccjjmartin’s picture

ccjjmartin commented
Issue summary: View changes
Status: Postponed » Closed (outdated)

This is available via the script run-tests.sh with the --xml option. Example command to run from the site root:

php ./scripts/run-tests.sh --xml "/var/www/sitename/junit_xml_directory/"