Running tests through command-line
Command-line test execution - Linux
This section is for the those that have the desire and/or the need to run the test-suite by command-line. This is often needed when one wants to integrate 3rd-party continuous-build tools, for example. I am hoping this page will collect info and tips on the manual aspects of testing, when one is doing its small-scale and only wants to test one or a few modules.
For those looking for info on the GUI way of running Simpletest, head for the Simpletest tutorial; lower down in that tutorial and you will find some information for running tests through the Drupal user interface. The two important links for command-line commands accessing Simpletest through Drupal interface are admin/config/development/testing/ and admin/config/development/testing/results/
Running the tests through the command line in Drupal is quite easy with the help of run-tests.sh, located in /scripts. One can easily get the test-suite running by going to the root of the site directory, as web-server user and typing php scripts/run-tests.sh.
If you don't feel like typing /scripts/run-tests.sh all the time, symlink the script to your ~/bin directory, assuming you have an existing bin directory that is in your path:
#Drupal 8 ln -s core/scripts/run-tests.sh ~/bin # Drupal < 8 ln -s scripts/run-tests.sh ~/bin
Running the script with no arguments will bring-up the help page, with a list of the available options.
Here is the run-down, shamelessly ripped from run-scripts.sh, note Drupal 8 users need to add core/ in front of scripts:
www-data@dev:/var/www-drupal-8-head$ php core/scripts/run-tests.sh Run Drupal tests from the shell. Usage: run-tests.sh [OPTIONS] <tests> Example: run-tests.sh Profile All arguments are long options. --help Print this page. --list Display all available test groups. --clean Cleans up database tables or directories from previous, failed, tests and then exits (no tests are run). --url The base URL of the root directory of this Drupal checkout; e.g.: http://drupal.test/ Required unless the Drupal root directory maps exactly to: http://localhost:80/ Use a https:// URL to force all tests to be run under SSL. --sqlite A pathname to use for the SQLite database of the test runner. Required unless this script is executed with a working Drupal installation that has Simpletest module installed. A relative pathname is interpreted relative to the Drupal root directory. Note that ':memory:' cannot be used, because this script spawns sub-processes. However, you may use e.g. '/tmpfs/test.sqlite' --dburl A URI denoting the database driver, credentials, server hostname, and database name to use in tests. Required when running tests without a Drupal installation that contains default database connection info in settings.php. Examples: mysql://username:password@localhost/databasename#table_prefix sqlite://localhost/relative/path/db.sqlite sqlite://localhost//absolute/path/db.sqlite --php The absolute path to the PHP executable. Usually not needed. --concurrency [num] Run tests in parallel, up to [num] tests at a time. --all Run all available tests. --module Run all tests belonging to the specified module name. (e.g., 'node') --class Run tests identified by specific class names, instead of group names. A specific test method can be added, for example, 'Drupal\book\Tests\BookTest::testBookExport'. --file Run tests identified by specific file names, instead of group names. Specify the path and the extension (i.e. 'core/modules/user/user.test'). --directory Run all tests found within the specified file directory. --xml <path> If provided, test results will be written as xml files to this path. --color Output text format results with color highlighting. --verbose Output detailed assertion messages in addition to summary. --keep-results Keeps detailed assertion results (in the database) after tests have completed. By default, assertion results are cleared. --repeat Number of times to repeat the test. --die-on-fail Exit test execution immediately upon any failed assertion. This allows to access the test site by changing settings.php to use the test database and configuration directories. Use in combination with --repeat for debugging random test failures. --browser Opens the results in the browser. This enforces --keep-results and if you want to also view any pages rendered in the simpletest browser you need to add --verbose to the command line. --non-html Removes escaping from output. Useful for reading results on the CLI. <test1>[,<test2>[,<test3> ...]] One or more tests to be run. By default, these are interpreted as the names of test groups as shown at admin/config/development/testing. These group names typically correspond to module names like "User" or "Profile" or "System", but there is also a group "Database". If --class is specified then these are interpreted as the names of specific test classes whose test methods will be run. Tests must be separated by commas. Ignored if --all is specified. To run this script you will normally invoke it from the root directory of your Drupal installation as the webserver user (differs per configuration), or root: sudo -u [wwwrun|www-data|etc] php ./core/scripts/run-tests.sh --url http://example.com/ --all sudo -u [wwwrun|www-data|etc] php ./core/scripts/run-tests.sh --url http://example.com/ --class "Drupal\block\Tests\BlockTest" Without a preinstalled Drupal site and enabled Simpletest module, specify a SQLite database pathname to create and the default database connection info to use in tests: sudo -u [wwwrun|www-data|etc] php ./core/scripts/run-tests.sh --sqlite /tmpfs/drupal/test.sqlite --dburl mysql://username:password@localhost/database --url http://example.com/ --all
Running a single simpletest class/method (Drupal 8)
To allow for faster iterations in your workflow, you may want to run a single test class or a single test method within a test class:
# Single test class php core/scripts/run-tests.sh --browser --verbose --url http://example.com/ --class "Drupal\module_name\Tests\TestClass" # Single test method php core/scripts/run-tests.sh --browser --verbose --url http://example.com/ --class "Drupal\module_name\Tests\TestClass::testMethod"
Please, read the code. This page just scratches the surface as the test-running script mentioned here can be extended and re-used at will.
- Do make sure you run tests as web-server user.
- If you are running tests on a CLI from remote shell access, the screen(1) utility can be handy for long running test sessions.
Command-line test execution - Windows (Drupal 7)
Windows users do not have a ready-made .bat file that would make things click and go. However, the majority of the rules applies for Windows as well. All the options above work just the same. The easiest way to use this script is having a Bash environment. Using either Cygwin, or MSYS that comes with Git for Windows are both ample enough to use the Linux notes above.
To use this script with strictly Windows CMD, navigate to your drupal root, run the script using php while specifying the php installation you are using as follows:
php scripts\run-tests.sh --php C:\PHP5\php.EXE "My Test Group Here"
Keep in mind your version of PHP may be somewhere different. Example if you are using XAMP or Acquia Dev Desktop, your php executable may be somewhere in those installations.
You can also find out what PHP Executable you are currently using (for php >= 5.4) by typing:
php -r "echo PHP_BINARY;"
Lastly, there is an issue for the redundancy of having to re-specify the php executable in Windows at.
Running tests without an installed Drupal site (D8)
In Drupal 7, in order to run tests there must be an installed Drupal site with Simpletest module enabled. In Drupal 8 it is possible to run tests without enabling the module, and without installing the site at all, by using an sqlite database which is created on the fly.
Running php ./core/scripts/run-tests.sh --help displays the syntax. For example, you can run tests with this command (issued from document root):
sudo -u www-data php ./core/scripts/run-tests.sh --sqlite /tmpfs/test.sqlite --dburl mysql://username:password@localhost/database --url http://example.com/ --all
Breaking this command down a bit:
Use sudo to set the user to the user which will execute php (on Debian or Ubuntu, often 'www-data'). On a VM you may need to first make yourself root with 'sudo su root'.
The --sqlite flag tells the test runner to create a temporary database for the test. The name and the folder where you can store it can be anywhere in the file system provided the web server can write to it. After the test, the sqlite database can be cleaned for reuse using the --clean flag, or manually deleted.
Even if there is no installed site the --dburl flag is needed. In order to avoid errors the database should exist (though it can be completely empty) and the username (with database password if you are using one) should have access to it.
The --url flag is also needed and should be a working URL on your server, linked in your webserver configuration to the document root where the code to be tested is. The script will run without a working URL but many tests will fail. In order to set up a working URL when developing locally it is normally also necessary to create a line in the hosts file to generate a working URL: on Linux to use http://example.com/, you might add
127.0.0.1 example.com to /etc/hosts. On a local install of the Drupal testbot using the VM image described here (and probably some other VMs) you may also need to adjust the network interfaces to get the URL working, unless you place or link your code in a subdirectory of the site which is already set up at http://172.16.2.100/ .
The --all flag will run all tests. Replace this with the relevant tests using one of the flags described by the --help flag. To test one class, use for example --class "Drupal\action\Tests\ActionUninstallTest". Note that in same cases testing an individual class without an installed database will fail with 'table not found'.
If you see an error like:
simplexml_import_dom(): Invalid Nodetype to importsimplexml_import_dom(Object)
This usually means the test runner's browser is not working. Make sure your web server is running and you have passed a correct
--url to the command.
If you have Drupal inside a virtual machine and access its web pages through a browser on your host machine, and you see an error like:
Test ExampleTests->setUp() failed: GET http://example.local/user returned 0 (0 bytes).
Even though you run your WebTestCases through the script or Drush, the virtual machine still needs to access 'example.local' (whereas for normal operation of the site it doesn't). So if you are pretty certain your code is not whitescreening at that point, try adding the domain name to your local hosts on the virtual machine.