Problem/Motivation
When a Panther PHPUnit run has errors or failures, the output is hard to read. Assertion failure messages include the full HTML page dump that Panther attaches to the exception, and there is no consolidated list of what failed at the end of the run, unlike the standard PHPUnit result summary. Two subscribers also record the wrong status: TestErroredSubscriber and TestMarkedIncompleteSubscriber both route their events through markTestFailed(), so errored and incomplete tests are reported as failures.
Steps to reproduce
- Run a functional JavaScript test that fails an assertion. The failure message contains the entire rendered HTML page, and no numbered error/failure summary is printed at the end of the run.
- Mark a test errored or incomplete, and observe it counted as a failure.
Proposed resolution
- Print a PHPUnit-style end-of-run summary listing errors, failures and warnings as a numbered list, each with a clean single-line message plus its stack trace, followed by a final test count line (
OK (N tests)orFAILURES! Tests: …). - Store the error message and stack trace separately on
TestInformationinstead of concatenating them, and strip the HTML page dump down to the first line. - Fix
TestErroredSubscriberandTestMarkedIncompleteSubscriberto record the correct status viamarkTestErrored()andmarkTestIncomplete().
Remaining tasks
Remaining tasks
- Review the merge request.
User interface changes
PHPUnit test output now ends with a numbered summary of errors, failures and warnings plus a test count line, replacing the inline HTML page dump in failure messages.
API changes
TestContext::markTestErrored(), markTestFailed() and markTestInconal <code>string $stackTrace parameter. FormatterInterface gainsresultSummaryHeader(), resultSummaryItem() and testCountSummary(). The renderMessage() helper in MessageTrait is removed in favour of separate getFirstLine() and renderStackTrace() calls.
Data model changes
TestInformation gains a string $stackTrace property.
Comments
Comment #2
lussolucaComment #3
lussolucaComment #5
lussoluca