It would be cool to have daily automated test runs of latest Drupal version dev and stable. This would mean that every day we would get report if latest HEAD or latest version of 6 branch is still passing tests.

CommentFileSizeAuthor
#3 simpletestauto_updated.patch40.49 KBcyberswat
#2 simpletestauto.patch39.96 KBcyberswat

Comments

cyberswat’s picture

cyberswat commented
Assigned: Unassigned » cyberswat

The current code base is slanted towards checking out the same version of the simpletest module as what is being tested. It performs a second checkout after getting the base version of Drupal it is testing and appears to run tests against this checkout. Is it desirable to follow this approach or should the simpletest modules be run in a stable version of Drupal that then interacts with the temporary test instance?

cyberswat’s picture

cyberswat commented
Status: Active » Postponed (maintainer needs more info)
StatusFileSize
new simpletestauto.patch39.96 KB

Waiting on some info from hunmonk to make sure I didn't break pift processing ... This patch allows for full testing of specific versions in addition to the existing patch testing. It also reports on testing results and reports complete failures. Once I speak with hunmonk I may make modifications before signing off on this but don't want to loose the work.

cyberswat’s picture

cyberswat commented
Status: Postponed (maintainer needs more info) » Needs review
StatusFileSize
new simpletestauto_updated.patch40.49 KB

Hasn't been run through coder yet but here is what I believe to be a working version

boombatower’s picture

boombatower commented
Status: Needs review » Fixed

Since #308974: Test all RTBC and CNR issues on rotation is working I think that alleviates the need for this as patches shouldn't be committed if they make tests fail.

c960657’s picture

c960657 commented

In practice, patches are sometimes committed, even if they cause tests to fail. Sometimes the tests only fail in certain environments (PHP version, MySQL version, Windows/Linux, $clean_urls, $db_prefix etc.). It would be useful if there was a number of machines with different configurations running tests on HEAD daily or weekly. Of course it would be optimal if all patches were run on all these machines prior to being committed, but a more modest solution that just pulls from CVS would be useful.

Examples of issues that are only present in some environments:

boombatower’s picture

boombatower commented

#5: The plan is to wait for testing results before committing, but that relies on all this stuff working. (very close)

As for machines with different OS/db engines, that will come later. Right now we just want to get all the features that are necessary working and stable. We are running into a few quirks.

Then eventually I see running a patch through multiple OSs and db engines and comparing results.

Anonymous’s picture

Anonymous (not verified) commented
Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for two weeks with no activity.