I've been spending a lot of time in provision_tests.drush.inc lately, and I have a couple small improvements to suggest. First off, we can strip out any Drupal5 tests, as it already isn't supported in Aegir 2.x anyway. Also, it can take a long time to run the entire test suite, so I propose adding an option to specify which tests to run: drush provision-tests-run --tests=migrate. It would default to running all tests, and should be pretty trivial to implement.

CommentFileSizeAuthor
#1 tests_cleanup.patch6.58 KBergonlogic
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

ergonlogic’s picture

Status: Active » Needs review
FileSize
6.58 KB

Essentially, like the attached patch.

anarcat’s picture

Status: Needs review » Needs work

I am not sure about all this. Do the Drupal 5 tests fail in Aegir 2 now? If not, why remove them? If we could keep on supporting D5, that would be great.

As for modular tests, it seems to me we should look at how Drush and simpletest are doing tests. It seems to me we are creating our own testing framework here and I would rather reuse existing code.

ergonlogic’s picture

I don't know whether the Drupal 5 tests pass or not, because they never run. Are we actually testing on a box with PHP <5.2 in Jenkins?

Either way, Drupal 5 is explicitely un-supported by Drush 5, which I believe is the new minimum recommended version for Aegir 2, no? Also, a Drush 6.x branch was recently started that requires PHP >5.3.5. It'll be the lowest Drush version to support Drupal 8, since that was just dropped from Drush 5.

As for how Drush handles testing, I'll look into it, but I'm under the impression that it's unit testing, rather than the functional testing that we're doing here.

ergonlogic’s picture

I just posted #1830220: Drop support for Drupal 5 in 2.x, since I think my arguments in #3 are outside the scope of just cleaning up testing. I'd happily drop the removal of the Drupal 5 tests pending resolution of that issue.

As far as I can tell, Drush doesn't provide a command to test itself. At least, I can't find a command to run the tests defined here: http://api.drush.org/api/drush/tests%21drush_testcase.inc/5.x. Also, the class Drush_TestCase, from which all other tests appear to inherit, mostly contains functions to set up a sandbox Drupal install, against which to test various Drush commands. So, I don't know how helpful it'd be. Also, they appear to depend on an external framework, PHPUnit, so I think we're on our own.

anarcat’s picture

To run drush tests, you use tests/runner.php or phpunit directly, as detailed in tests/README.txt in the Drush distribution.

Using phpunit seems like a better way to go than creating our own framework. Either we use phpunit or we use simpletest, but if we are to refactor things, i would rather avoid reinventing the wheel.

Adding a dependency (phpunit, in this case) to the test suite isn't a problem for Drush and shouldn't be for us either.

I agree with you for Drupal 5 now. :) Feel free to just remove them (instead of making them optional) in 2.x.

Steven Jones’s picture

Title: Clean up and modularize tests » Re-work tests to use PHPUnit and Drush's testing framework
ergonlogic’s picture

Version: 6.x-2.x-dev » 7.x-3.x-dev
Issue summary: View changes
Issue tags: +testing initiative

New features need to be implemented in Aegir 3.x, then we can consider back-porting to Aegir 2.x.