I've come up with a way to generate code-coverage reports for simpletest runs without having to patch core (7.x). I maintain a working PoC in a sandbox. I'd like to illustrate the main ideas here and hope to gather some feedback.

Inject setUp and tearDown using namespaces and autoload

Regrettably simpletest does not provide any hooks suitable for instrumenting test-cases. Therefore another technique is employed to inject the necessary code into the test classes. Using hook_simpletest_alter. Every test-class is prefixed with the namespace CoverSimpletestWrap. E.g. instead of ActionLoopTestCase simpletest will try to instantiate CoverSimpletestWrap\ActionLoopTestCase when attempting to run that test case. This class is then constructed on the fly from within an autoload-function overriding the setUp and tearDown methods of the original test-case.

Start recording when destination directory is present

A recording is only started when the destination directory (a.k.a. session_dir) is present and writable. This is a very simple check doable very early in the page request - even from within the front controller (index.php) before calling drupal_bootstrap. In order to support parallel execution of test-cases, the session_dir can be changed using the X-Cover-Session-Dir. The header value is protected using a HMAC just like the simpletest user-agent string.

When collecting coverage data is not desired, we simply do not create the destination directory in the first place.

Autostart recording in simpletest child-site

When generating code coverage reports for non-bootstrapping modules, there is no need to start recording data earlier than from hook_boot. Therefore when setting up web test cases a tiny helper module is enabled in the child-site which simply starts recording coverage data from within a hook_boot implementation. For many contrib-modules this is enough and therefore in this cases neither modifications on index.php nor on settings.php are necessary. Just install PHP Code Coverage from PEAR and enable cover_simpletest is enough.

Comments

djdevin’s picture

djdevin
Location Philadelphia
CreditAttribution: djdevin commented

I've tested this sandbox module and it works pretty well so far.

Definitely something to look into.

alberto56’s picture

alberto56 CreditAttribution: alberto56 commented

@znerol, thanks for sharing! From what I gather, your module and the Code Coverage module use different approaches and aim to solve different problems. Code coverage gives you on the fly reporting for any page you visit, whereas your module provides a report for tests. I'm wondering what the maintainers of code coverage would prefer, but I would definitely release your module, it works exactly as advertised, is very intuitive and provides useful information from the get-go. I have used it in conjunction with tests on a site deployment module, so I can provide feedback to my team, employers and clients about exactly how much code we are testing.

Thanks again!

trogie’s picture

trogie CreditAttribution: trogie commented

very nice indeed!

The only thing I (or somebody) want to change is that reports can be generated independent from 'running tests' and that there is a drush command.

With a drush command this report generation could be easily automated.

Edit: not too much interested to test the 'code coverage' as it requires a core patch with will always remain a little 'dirty' to upgrade, have real dev/stage/prod environments,...