FAQ

Last updated on
22 March 2021

How do I enable automated testing for my project?

Make sure the project has a dev release. From the project page, pick the "Automated testing" tab. Select the frequency of testing you want next to the environments you want tests run in. (Handbook page for enabling automated testing.). If you have a distribution project, there is no "Automated testing" tab, because we don't support distribution testing yet.

Why is my patch marked "postponed"?

Most of the time this is because the branch you're testing against hasn't passed tests. From the project page, pick the "Automating testing" tag, and look under "Current results". Click on the testing results to see what the fails are. Usually some work is required on the branch (and tests won't succeed without new commits).

Why does my test fail on the testbot but works OK locally?

There are probably three key differences between your local test environment and the testbot

  • The testbot is running a different version PHP than your local. Check what version the test ran on by clicking the test results link, then "View results on dispatcher", then "View as plain text", and then search for "php -v" to find the php version in the test output.
  • The testbot has clean URLs turned off by default (as does Drupal)
  • The testbot webroot is in a subdirectory (like http://drupaltestbot.osuosl.test/checkout/?q=admin)
  • The testbot runs tests using run-tests.sh. Try using that technique and see if your tests succeeeds
  • The testbot has to be able to run through the install process, then log in and enable simpletest and visit the testing page, all using the web UI. Have you tried that?
  • The testbot starts with all modules disabled and the tests themselves enable them. Disable your relevant modules so that they're not active locally before running local tests to get a reasonable result.
  • The testbot uses the issue version to determine which codebase to test against. Ensure the issue the patch has been uploaded to is set to a -dev release, as opposed to a specific release checkout.
  • If you get "failed to apply", check that the issue is set to the *branch*, not a specific tag. For example, an issue with a patch should be set to "8.0.x-dev", not "8.0.x-rc2". Try to apply the patch to a local repository just to make sure. If it fails to apply on the testbot, it will fail locally too, if you're using "git apply".
  • If installation is failing, please try installing Drupal with your patch applied. Your patch may cause a failure in the installation.

I changed dependencies in my .info file, but the wrong dependencies are still being loaded on test

Dependencies only get built when the new dev release is rolled, every 12 hours. So if you change dependencies, it won't be reflected until the new dev tarball is rolled.

I'm pretty sure a testbot is misbehaving (Can't drop database... Can't clear checkout directory... Hasn't done a test in a few hours...) What should I do?

See Testbot broke! What do I do now?

My patch failed due to a failure in an unrelated test, but when I retested it, it passed. What should I do?

If you think the first failure might have been testbot related, then open an issue in the Drupal CI project or one of the project listed as related there.

If it turns out to be a problem with the test itself intermittently failing, we'll tag it with Random test failure and bounce it to the core issue queue.

Where can I find test output?

Click through from the issue testing area or Automated Testing page to the test results. Click View results on dispatcher. There you will find the test output (see Understanding test output for more details).

For contributed project testing, you can find the HTML output generated within browser-based tests under:
build artifacts > (directory specific to the test) > artifacts > run_tests.standard > simpletest_html
or (for JavaScript-based browser tests):
build artifacts > (directory specific to the test) > artifacts > run_tests.js > simpletest_html

Due to the number of tests in Drupal core, this output is not saved for Drupal core.

Help improve this page

Page status: No known problems

You can: