This patch extends on the work done for http://www.computerminds.co.uk/hudson-and-simpletest and adds an --xml output option for run-tests.sh that works with SimpleTest 6.x-2.9.

Files: 
CommentFileSizeAuthor
#22 602332-xml-reporter.patch6.33 KBSteven Jones
PASSED: [[SimpleTest]]: [MySQL] 26,779 pass(es).
[ View ]
#21 602332-xml-reporter.patch6.33 KBSteven Jones
PASSED: [[SimpleTest]]: [MySQL] 22,851 pass(es).
[ View ]
#10 602332-xml-reporter-10.patch5.32 KBpwolanin
PASSED: [[SimpleTest]]: [MySQL] 22,171 pass(es).
[ View ]
simpletest-junit-xml-output.patch4.71 KBSteven Merrill
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch simpletest-junit-xml-output.patch.
[ View ]

Comments

joshk’s picture

+1 for this

This kind of output will easy integration with Hudson and other external test mgmt systems.

I also like that this shows how to do it in a more modular fashion with an extensible pattern.

ezraw’s picture

Tested this patch tonight and it output xml as expected. Thanks for posting it. I haven't tested it with Hudson yet, but that's next on the list. If you have any helpful tips on doing so, I'd be grateful.

ezraw’s picture

Status:Needs review» Reviewed & tested by the community

The XML output works with Hudson for me.

I have about 200 test errors, some from core functionality -- is that normal? Here is the Hudson output: http://drupalbin.com/11879

boombatower’s picture

This should be done in Drupal 7 core...since it is going into feature freeze not sure what to say. SimpleTest 6.x-2.x is intended to be a backport of 7.

boombatower’s picture

Version:6.x-2.9» 7.x-2.x-dev

We are continuing feature development, as suggested by chx (checkout recent Drupal Planet activity).

I'll take a look at this later, also I would like to abstract the reporter.

Steven Jones’s picture

Given that I wrote the script referenced in the first post, subscribe.

Status:Reviewed & tested by the community» Needs work

The last submitted patch, simpletest-junit-xml-output.patch, failed testing.

pwolanin’s picture

is there an issue for core?

pwolanin’s picture

Project:SimpleTest» Drupal core
Version:7.x-2.x-dev» 7.x-dev
Component:Code» simpletest.module
Assigned:Steven Merrill» Unassigned
Category:feature» task

Let's get this into core - it's just addition of another output format to the wrapper script to improve integration possibilities.

pwolanin’s picture

StatusFileSize
new5.32 KB
PASSED: [[SimpleTest]]: [MySQL] 22,171 pass(es).
[ View ]

Here's a patch that seems to work for D7 with some cleanup.

pwolanin’s picture

Status:Needs work» Needs review
boombatower’s picture

/me wonders when feature development of Drupal 7 actually stops.

pwolanin’s picture

boombatower - this is not anything I'd qualify as a real "feature". In's not really part of drupal core, jsut a helper script. Putting is in this script just means people don't have to fork the script to get more output formats

moshe weitzman’s picture

If I'm not mistaken, PIFR has some home grown XML output (or it did in v1) so this would deprecate that code as well. +1

Steven Jones’s picture

Is it possible to test this? Guessing that we can't run tests in tests to ensure that the output is the correct, valid JUnit XML?

boombatower’s picture

Please take a look at simpletest.test.

pwolanin’s picture

Apparently a number of people are already using the 6.x versionof this patch (which is in pressflow) to integrate with Hudson.

I think that's basically the test - can you get Hudson to parse and report the xml output.

Steven Jones’s picture

If only a schema for JUnit output existed...oh well.

Guess we'll need someone to plug it into Hudson then...

Steven Jones’s picture

Assigned:Unassigned» Steven Jones

Probably should do this sooner rather than later...will spin up hudson tomorrow and get him testing.

Steven Jones’s picture

Assigned:Steven Jones» Unassigned
StatusFileSize
new6.33 KB
PASSED: [[SimpleTest]]: [MySQL] 22,851 pass(es).
[ View ]

This is working nicely with Hudson. The result is nice and valid.

I've added exception reporting too in the attached patch.

Steven Jones’s picture

StatusFileSize
new6.33 KB
PASSED: [[SimpleTest]]: [MySQL] 26,779 pass(es).
[ View ]

Actually, we'll use the error element for exceptions, not the failure one.

nikgregory’s picture

Status:Needs review» Reviewed & tested by the community

I ran this on an ubuntu install and it seems to work well.

seanberto’s picture

Sorry if it's a n00b question, but I can't seem to get run-tests to output to an xml file.

* I've run the patch in #22 on 7.0-alpha7.
* I've created a tests directory at scripts/tests. I've set the permissions on this directory to be globably writable.
* From within that directory, I'm running the shell command:

/usr/bin/php run-tests.sh --url http://example.com --all --php /usr/bin/php --xml scripts/tests

I see the output in my terminal session and the tests are running - but nothing is written to this directory. I've tried making the path to the directory relative to scripts as well, but no luck.

I'm guessing that there's something easy that I'm missing?

Thanks for the help. I'm excited to have Hudson up and running and building D7 successfully. I'll be even more stoked when I can help with automated testing!

Cheers,
Sean

seanberto’s picture

Aha, solved my own issue. It was a permissions oversight. Gotta make sure that scripts/tests directory is writable by Drupal.

sun’s picture

Version:7.x-dev» 8.x-dev

Although badly needed, this sounds like D8 material to me.

pwolanin’s picture

Version:8.x-dev» 7.x-dev

Don't be silly. This has no impact on core's functionality.

sun’s picture

Version:7.x-dev» 8.x-dev

Trust me, I know. But Drupal core maintainers only take actually required bug fixes into account for D7. Everything else is D8 material for now. I'm with you. I had to learn the hard way.

pwolanin’s picture

Version:8.x-dev» 7.x-dev

I think this is still suitable for D7, though obviously critical issues come before.

Steven Jones’s picture

#22: 602332-xml-reporter.patch queued for re-testing.

dasjo’s picture

would be great to get it in!

dixon_’s picture

subscribing

nooble’s picture

#22: 602332-xml-reporter.patch queued for re-testing.

tom_o_t’s picture

simpletest-junit-xml-output.patch queued for re-testing.

Dries’s picture

Status:Reviewed & tested by the community» Fixed

Committed to CVS HEAD. Thanks.

seanberto’s picture

Hurray!!!!

Status:Fixed» Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.