We would appreciate it is only project maintainers commented on this issue.

Please list the projects you maintain that you would like to be part of the beta stage and if they currently have any tests. For example if I were to comment requesting the subuser module to be added:

subuser - has tests

We will contact you if any of your projects are included in the beta stage.

Thank you.

UPDATE: Currently enabled contrib project (more to come - as we fix bugs and such).

3060    drupal
3236    devel
3256    image
3277    poormanscron
3279    privatemsg
5461    weather
8404    captcha
11873   simpletest
17345   pathauto
19304   media
23158   comment_notify
41542   porterstemmer
59060   taxonomy_filter
65961   geshifilter
106016  token
108746  admin_menu
109640  services
128621  adminrole
129418  potx
138844  autoassignrole
144819  content_access
148272  l10n_server
177400  openlayers
190124  rules
190839  xmlsitemap
204326  talk
240806  mollom
243795  linkchecker
253481  storm
287853  languageicons
313122  project_issue_file_review
333543  role_change_notify
441778  search_by_page
476696  og_statistics
477446  grammar_parser
511618  faces
520254  nd
543396  encrypt
571608  password_change
594964  examples
597564  blogapi
623208  profile2
626176  entity
689342  proxy
691808  contact
730762  simpletest_selenium

and vars, google_analytics, versioncontrol, vote_up_down, opensearch, transformers, libraries, multicron, smartcrop, edge, translation, user_delete, piwik, embeddable, skinr, rtsg, profile_permission, userpoints, userpoints_nc, userpoints_contrib, chessboard, css_injector, uuid, nodequeue, properties, date_popup_authored, references, taxonews, g2, fontyourface, myspace_sync, field_collection, mailhandler, enabled_modules, commerce, cck_signup, ubercart, restws, views, cas, current_content.

Thanks to people reporting issues we were able to fix the following issues and deploy the changes, I'll be enabling more contrib projects now. (01/28/10)

PIFR 6.x-2.2, 2010-01-27
------------------------
- Bugs:
   * #695350: Provide 'last' field in pifr.retrieve() and correct query.
   * Events should only be triggered when trigger modules is available.
   * Remove notices when test record is saved before client record.
   * #695278: Test list should be generated from root directory.
   * #695278: Module 'tests' directory should be searched.
   * #696044: Patches are not being applied properly to contrib projects.
   * #696194: Cannot preview client test information.
   * Do not add SimpleTest as dependency if it has already been added.

Comments

cyberswat’s picture

autoassignrole - has tests

Edit by Dave: confirmed testing enabled

Dave Reid’s picture

XML sitemap - 100% test suite pass (and technically is already enabled for testing on drupal.org)
Admin role - 100% test suite pass
Poormanscron - 100% test suite pass
Password change confirm - 100% test suite pass
Examples - would be great to make sure they're current with development

Edit by Dave: confirmed testing enabled

Dave Reid’s picture

Contact - backport/experimental replacement for core contact.module

Edit by Dave: confirmed testing enabled

sun’s picture

http://drupal.org/project/admin_menu - has tests
http://drupal.org/project/mollom - has extensive tests

Edit by Dave: confirmed testing enabled

Berdir’s picture

Privatemsg - has tests (Drupal 6 and 7)

Edit by Dave: confirmed testing enabled

fago’s picture

Content Access has tests
Rules has tests (many more in d7)
Entity API has tests (has two different modules to test, does that work?)
Faces API has tests
Profile2 has tests

Edit by Dave: confirmed testing enabled

Dries’s picture

Can we add Mollom, please? We have many tests for both D6 and D7.

joachim’s picture

http://drupal.org/project/image - has tests

Edit by Dave: confirmed testing enabled

toddy’s picture

http://drupal.org/project/weather - has tests

Edit by Dave: confirmed testing enabled

hass’s picture

http://drupal.org/project/linkchecker also have tests and may show many coder bugs. I have for sure found many false alarms in past and they are not fixed as I know.

Edit by Dave: confirmed testing enabled

heyrocker’s picture

http://drupal.org/project/services - has no tests but we're getting ready to start rolling a stable release and this would prove really helpful with handling bug fix patches.

Edit by Dave: confirmed testing enabled

zzolo’s picture

http://drupal.org/project/encrypt - has tests
http://drupal.org/project/proxy - has tests (though failing at the moment)

Edit by Dave: confirmed testing enabled

aaron’s picture

http://drupal.org/project/media -- still in development; has tests

jhodgdon’s picture

Not sure how many you want here, but
http://drupal.org/project/porterstemmer has tests
http://drupal.org/project/search_by_page has tests, and several of them depend on other contrib modules

Gábor Hojtsy’s picture

cwgordon7’s picture

talk - has tests (DRUPAL-6--1 branch)

soxofaan’s picture

captcha - has tests
geshifilter - has tests

swentel’s picture

Magnity’s picture

http://drupal.org/project/storm - has 1 basic test, with more to come soon.

Freso’s picture

http://drupal.org/project/languageicons - No tests (yet), but I certainly wouldn't mind being included in this beta. =)

joachim’s picture

The issue description has been updated, and seems to imply the contrib testing bot is now live. Do existing patches that were already set to 'needs review' need prodding in some way for the bot to notice them? -- they still say 'ignored'. Eg: http://drupal.org/node/412288

boombatower’s picture

The system does not cycle in all previously existing patches, adding a comment of any kind to the issue will trigger a test. Also making a commit will trigger the branch test. If the branch test does not pass, the patches will be postponed.

Parsing all previously existing patches is not currently supported.

zzolo’s picture

For Proxy (listed above as being enabled), I am not able to get the testing to start for patches and it still says ignored; for example:
http://drupal.org/node/699180#comment-2548690

Also, will there be an easy way to find the branch testing, like a link on the project page?

Thanks for the great work on this. It is very awesome!

Magnity’s picture

Are there any plans to add more contrib projects to the beta stage now that the testbot is back up and running?

Thanks.

Dave Reid’s picture

@zzolo You have to make a commit to the desired branch to get it 'enabled' for testing.

dawehner’s picture

http://drupal.org/project/og_statistics - has tests for every feature.

ilo’s picture

Cweagans and I are going to have several sprints on blogapi module. I'm not the module mantainer, however cweagans asked me to do this request because he was not going to be able to do for the next days as commented in the issue:#686832: Split connection methods into abstracted backends

So, our interest is to have the blogapi module included in the contrib testbot.

Thanks!

rfay’s picture

Please add role_change_notify to the test, as it now has tests.

zzolo’s picture

openlayers 2.x branch now has a number of tests.

Freso’s picture

Pathauto's HEAD finally also has some proper tests, so it would be good to have that included in this as well. :)

greggles’s picture

Pathauto HEAD and 6.x* branches are already included - http://qa.drupal.org/pifr/test/31923

comment_notify - has tests that pass on 6.x branch. The 7.x tests for comment notify are currently broken.

drumm’s picture

API module's tests will cleanly pass when #735724: Remove globalToString() echo is resolved. However, I'm guessing an automated 6.x checkout of Grammar Parser will fail due to the "Looking for 6.x support?" note at http://drupal.org/project/grammar_parser.

tstoeckler’s picture

Clock - has tests for 6 and 7.

Steven Jones’s picture

Context - has tests

Steven Jones’s picture

Ah, sorry can you remove the context project, we depend on a lot of contrib modules, and as simpletest gets the wrong versions a lot of our tests fail when they really shouldn't. Apologies for time wasted.

boombatower’s picture

@Steven Jones: please check back when you see another front page post about an update as it should provide a way to specify versions.

gordon’s picture

Can you please also add ecommerce http://drupal.org/project/ecommerce

JoshuaRogers’s picture

plugin_manager - has some tests. More are coming.

grendzy’s picture

janusman’s picture

boombatower’s picture

Added last four.

mikeker’s picture

Thanks to you and Kieran at the QA sprint, I now have a meager handful of tests for my module. More to come!

Better Exposed Filters - has tests

janusman’s picture

Question: So... by just marking an issue as "needs review" in my issue queue, that enough to get tested? Nothing looks different when I do (see http://drupal.org/node/468388#comment-2874104 )

**EDIT** nevermind, found it =) [Thanks @berdir]

Anonymous’s picture

Variables API — has tests.

hass’s picture

Please add Google Analytics, too.

Could someone give me a small hint, please - why the patch in http://drupal.org/node/249701#comment-2979452 is "ignored" and not tested against 6.x-2.x?

Gábor Hojtsy’s picture

l10n_server which is included in the beta stage has issues with how the testing framework is set up vs. the tests being run: #802034: l10n_server testing fails due to potx not being there/not enabled

marvil07’s picture

Version Control API - has tests
Vote up/down - has tests, few and a little broken but including it will let me fix it quicker ;-)

jhodgdon’s picture

RE #48: I think the project issue testing module might have a bug left over from the past where it used to ignore patches with a -D6 suffix. You might want to file an issue on the Project Issue Testing project.

Anonymous’s picture

Opensearch feed — has tests.

kenorb’s picture

subscribe

sepgil’s picture

Rules Transformers has some few tests, but I'll add more, as the project advances.

tstoeckler’s picture

Libraries API now comes with tests! D7 version only, for now.

dstol’s picture

Multicron has tests.

grendzy’s picture

smartcrop - has tests

sun’s picture

http://drupal.org/project/edge comes with tests, too

sun’s picture

http://drupal.org/project/translation could use the testbot, too :)

ilo’s picture

Hi, http://drupal.org/project/user_delete (an attempt to backport D7) account cancellation module is including tests now.

Thanks!

Anonymous’s picture

Book access - has tests.

hass’s picture

fago’s picture

Jacine’s picture

ilo’s picture

Boombatower, I kindly ask you to include http://drupal.org/project/rtsg in the testing framework :)

ilo’s picture

Modules that would love to be integrated in the next round..

http://drupal.org/project/userpoints
http://drupal.org/project/userpoints_nc
eventually http://drupal.org/project/userpoints_contrib can be also included (due to the D7 upgrade on the way, but not sure if d6 has any testcase)

Eric_A’s picture

Chessboard Renderer - has tests
88 passes, 0 fails, and 0 exceptions

rfay’s picture

Please add css_injector.

ilo’s picture

uuid has tests now. Thanks!

ezra-g’s picture

Nodequeue has tests: http://drupal.org/node/468136#comment-3816526 - Thanks!

Berdir’s picture

Properties - has tests

Thanks :)

zzolo’s picture

This is really great! Thanks for all the hard work on this. It really adds lots of value to developing on d.o. I am curious if there is a discussion, or what the status is, on getting automated Coder reviews for the contrib space as well (as I know it is available for core)?

boombatower’s picture

We had a halt declared on all bot improvement while Drupal 7 was attempting to stabilize for final release...now that Drupal 7 is out development when continue...stay tuned.

zzolo’s picture

Thanks @boombatower for all the great work. If there's some way I can help, please let me know.

Mark Trapp’s picture

Not sure if you're still accepting submissions here, but Date Popup Authored has tests.

sun’s picture

For some time, I'm having serious trouble with the testbot in two contributed projects:

  1. Libraries API: Testbot throws an "Detect invalid PHP syntax" for the HEAD branch, but the review log details state a very weird error message:
    [21:12:08] Invoking operation [syntax]...
    [21:12:08] No such module [example_info_file.libraries].
    [21:12:08] Checked [17 of 17] relevant file(s) for syntax.
    [21:12:08] Encountered error on [syntax], details:
    array (
      '@reason' => 'No such module [example_info_file.libraries]',
    )
    

    The XML-RPC output also lists that example_info_file.libraries module, which does not really exist (an .info file with that name exists for testing purposes, but no module).

    Tests are passing successfully when being run locally.

  2. Skinr: Testbot passes on HEAD, but a current patch against HEAD does not pass, and after adding extra debug() output to the patch, the test results show a weird list of enabled modules -- neither the Skinr module itself nor the required Node module does not seem to be installed in the test environment.

    Here, also, tests are passing successfully when being run locally.

    Not sure whether this is related, but the XML-RPC output for that test shows that the testbot added a bogus additional dependency on the Skinr DRUPAL-7--1 branch (which is empty though).

I'm sorry for posting this in this issue, but I have absolutely no idea what the proper place for reporting such issues is. In particular, it's very hard to figure out whether this is an issue with SimpleTest, PIFR, PIFT, or some other component of the testing framework. In order to go out of beta with contributed module testing, we definitely need a single dedicated project issue queue to handle such issues.

Berdir’s picture

I have a very similiar issue with Userpoints. Sometimes, the testbot runs "Rules Scheduler" tests additionally to our own, producing unrelated test failures. See #945622-2: 'Reason' column: Description gets replaced by truncated operation when you edit point transactions created by other modules for an example.

Anonymous’s picture

@sun: Basing on the URL used for the test log pages (eg, http://qa.drupal.org/pifr/test/110619), I guess the project for which such issues should be reported is PIFR. AFAIK, PIFT is the module that integrates a server running PIFR with drupal.org.

I have created the issue #1023220: Tests enable extra, unrelated modules (and more).

fgm’s picture

Can you please add

- Reference (tests on D7 only)
- Taxonews (tests on both D6 and D7)
- G2 (tests on D5 and D7, not D6)

greggles’s picture

Title: Contrib projects to be included in beta stage » Contrib projects to be included in beta stage of automated testing for contributed modules

better title.

greggles’s picture

Title: Contrib projects to be included in beta stage of automated testing for contributed modules » Contrib projects to be included in beta stage of automated testing for modules

And now with 100% less repetitiveness and redundancy.

sreynen’s picture

fago’s picture

field-collection - has tests.

ilo’s picture

In the time you read this comment I'll be submitted the testcase for the Mailhandler project.

Thanks in advance!

catch’s picture

Entity cache - has a test stolen from core.

juliangb’s picture

Enabled Modules (http://drupal.org/project/enabled_modules).

Tests to be added soon.

rszrama’s picture

http://drupal.org/project/commerce - has hella-tests, still adding more... poor pcambra's lappy takes an hour and a half to run 'em all.

jhedstrom’s picture

http://drupal.org/project/cck_signup has tests (in 7.x only).

TR’s picture

http://drupal.org/project/ubercart - has about 1000 tests on each of the D6 and D7 branches, and more coming soon. Thanks!

TR’s picture

Are the tests being run taken directly from the repository, from the latest development snapshot, or from the latest tagged release of the contrib project?

Dave Reid’s picture

@TR Tests are run based on the version specified in the individual issues. It also runs 'branch' tests when you commit code to a branch, but those results are not displayed on d.org. They're only used to determine if it can continue to run tests on issues or if it needs to postpone tests.

rszrama’s picture

If I'm not mistaken, it's not working if you have dependencies, right? i.e. Commerce wouldn't work, so I'm guessing Ubercart won't either?

Dave Reid’s picture

Yes, if your module has dependencies on other projects, testing does not currently work very well. You can work around it to use testing as a 'does this patch apply or not', but not currently as a reliable testing framework.

rfay’s picture

I do hope we'll be able to get loading dependencies to work in the next month or two (in the near term). Fixing the criticals on PIFR is my biggest community priority. It's just that there are a lot of them. And boombatower already made this work (I think) in the mainline (master), but getting that deployed is still a pain.

rfay’s picture

BTW: I filed #1093886: Allow 'pift enable project testing' permission for project owners/enabled users to hopefully let project owners manage this themselves.

plach’s picture

Title has a couple of tests and we expect to have more in the near future ;)

klausi’s picture

restws has tests.

Berdir’s picture

User relationships has some tests, more to come (at least in D7)

kenorb’s picture

drupal.org/project/views please

merlinofchaos’s picture

kenorb: Thank you for impacting a busy development day by having this enabled on my module without my knowledge or permission.

In case you didn't realize, PIFR will automatically mark issues 'needs work' if the patch causes tests to fail. That means if the tests aren't perfect, perfectly valid patches get thrown into the 'needs work' graveyard and nobody knows about it. Thanks to you, I'm now stuck trying to figure out what pifr just screwed up for me.

For the love of all that is holy if this framework is enabled on a module, make ABSOLUTELY SURE the primary maintainer of the module consents, because if things aren't set up right, it is destructive.

rszrama’s picture

Not to mention the test bot won't really work for Views since it has a dependency on Ctools. :-/

kiamlaluno’s picture

I think that this issue is thought to be used by maintainers to request automated testing on projects they maintain.
kenorb should not have asked to enable them for Views, as he doesn't even have write access to Views repository.

rfay’s picture

/me was already long ago adequately lashed and repentant :-)

kenorb’s picture

@merlinofchaos: Sorry, if that cause any problem. I could help with test fails, I'm currently working on some of them.
I found only one issue with Views test
#1101332: Call to a member function set_display() on a non-object in tests/views_exposed_form.test on line 34
which I can solve.

Secondly enabling modules for test bot should be part of project configuration, not by posting module names in the issue. Especially when people don't know exactly how this test bot works, like me.

boombatower’s picture

It is a project setting, but currently only a few admins have the permission to enable. There is an issue in the infra queue to enable for all, but I believe it was decided to continue waiting on #102102: Parse project .info files: present module list and dependency information. The intention was not to leave it for this long, but a lot of things got in the way of getting this fixed up.

bfroehle’s picture

The CAS module has test routines. Please enable testing (for D7).

jn2’s picture

I have a project used as a practice space for making and reviewing patches. Would like to enable testing so that participants can also experience what that's like.

Current content is the module. Already has test routines. Please enable D7 testing.

fgm’s picture

Can you please enable MongoDB ? It already has some tests in some branches.

bfroehle’s picture

Please enable testing for CAS Attributes.

Thanks!

I'm looking forward to being able to enable testing myself as a module maintainer in the future!

rfay’s picture

CAS Attributes enabled

Niklas Fiekas’s picture

Media Feeds - has tests

rfay’s picture

Media Fields enabled.

greggles’s picture

I'm not sure that simpletest can handle modules with dependencies which I imagine Media Feeds has.

Niklas Fiekas’s picture

Thanks. Yes, it doesn't work. rfay pointed me to #102102: Parse project .info files: present module list and dependency information in IRC.

chx’s picture

May I get Relation? We do not have dependencies. We work with everyone but we do not depend on them.

xjm’s picture

Taxonomy access has the beginnings of tests in the 7.x branch, and they'll be backported to 6.x eventually.

rfay’s picture

Relation and taxonomy_access are turned on.

dstol’s picture

taxonomy menu now has tests.

rfay’s picture

Taxonomy Menu enabled.

fgm’s picture

Please add Classified Ads, which already has D6 tests which will be ported to D7.

rfay’s picture

Classified Ads enabled.
The D7 version has lots of files[] in the .info which are not necessary (they're only needed for files that contain classes).

TR’s picture

Can you please add http://drupal.org/project/ip2country ?

This project has 200 tests that run green for D6, D7, and D8 (!!!).

rfay’s picture

ip2country enabled.

adooo’s picture

Access by Term has tests...
Thanks...

rfay’s picture

Access by term enabled.

Niklas Fiekas’s picture

Please enable feeds_namespaces. It has dependencies, but the unit tests should work regardless.

rfay’s picture

feeds_namespaces enabled.

rmiddle’s picture

Please enable http://drupal.org/project/commentcloser. It has no dependencies, and the unit tests should work 100%.

rfay’s picture

done

Niklas Fiekas’s picture

Please enable http://drupal.org/project/birthdays. Has tests.

rfay’s picture

Birthdays added

ParisLiakos’s picture

Is there still a problem with modules that have dependencies, since #1241704: Deploy Project Dependency, Project and PIFT on d.o is marked as fixed?

If so are there any extra steps for modules that have dependencies before joining automated tests?
Thanks

boombatower’s picture

#1093886: Allow 'pift enable project testing' permission for project owners/enabled users which needed #1299910: Remove 'Enable Automated Testing' checkbox from projects without releases

Since it has now been completed it just needs to be deployed to allow project maintainers to enable testing themselves.

rfay’s picture

@rootatwc if you need anything enabled for testing, just say the word (until maintainers can do it themselves)

ParisLiakos’s picture

Great!
thanks boombatower for the fast response:)

@rfray http://drupal.org/project/newsletter has some tests and i am currently writing more.

boombatower’s picture

Enabled newsletter.

plach’s picture

rfay’s picture

enabled entity translation

TwoMice’s picture

rfay’s picture

civicrm_activity_ical enabled

fgm’s picture

The mongodb tests always fail because there is no /visible/ MongoDB server to which they can connect. I wonder if it really makes sense to have them run in automated mode at all except for making sure the patches are well-formed.

Now, is there not a MongoDB server in the d.o. infra somewhere, which could be accessed by these tests ?

rfay’s picture

Nothing I know of. You might want to add a condition to your test to skip tests if it can't connect to a server.

rfay’s picture

Status: Active » Fixed

Hooray! We no longer have to do this. Every project owner now has the capability to turn on testing. You do it by editing the project, issues tab "Enable automated testing".

See http://jthorson.doesdrupal.com/node/34 for more information.

Also please note that issues on the testbots and testing can always be filed in the Testbot queue.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

smokris’s picture

I couldn't find documentation for this functionality in the contrib module maintainers guide, so I added http://drupal.org/node/1449736.

rfay’s picture

@smokris++

Thanks so much.

Project: Drupal.org infrastructure » Drupal.org Testbots
Component: qa.drupal.org » Miscellaneous