Running PHPUnit tests

Last updated on
4 August 2017

PHPUnit tests are not well-integrated with the SimpleTest administrative interface and should be run from the command line.

Make sure to have the development dependencies installed

If you installed from a Drupal.org package and need the development dependencies, you have three options:

  1. Install composer and run composer install --dev.
  2. Use a development snapshot (for example, 8.2.x-dev) instead of a tagged release for your development site.
  3. Install the development dependencies you need manually into Drupal's vendor directory or elsewhere.

Remember that development dependencies should never be installed in production or a on publicly accessible server for security reasons. See the announcement about our dev dependency packaging changes for more information.

Run all PHPUnit unit tests

To run the unit tests on OS X, Linux or other *nix systems:

cd core
../vendor/bin/phpunit --testsuite=unit 

Note: All tests, including those located in [drupalroot]/modules or [drupalroot]/sites/*/modules, are run from the core folder with the ../vendor/bin/phpunit command

Note also that you don't need to have a working Drupal installation to run PHPUnit-based unit tests this way. PHPUnit tests are isolated from Drupal and don't need it in order to run.

On Windows, the symlink stored in core/vendor/bin/phpunit will not work. You need to use the full path to the phpunit executable:

cd core
../vendor/phpunit/phpunit/phpunit

Run kernel test and browser tests

For kernel tests you need a working database connection and for browser tests your Drupal installation needs to be reachable via a web server. Javascript browser tests require a running PhantomJS in addition.

Copy the phpunit config file:

cd core
cp phpunit.xml.dist phpunit.xml 

Fill in SIMPLETEST_DB, SIMPLETEST_BASE_URL, BROWSERTEST_OUTPUT_DIRECTORY and uncomment the printerClass property in the phpunit tag. The result should look something like this:

<phpunit bootstrap="tests/bootstrap.php" colors="true"
         beStrictAboutTestsThatDoNotTestAnything="true"
         beStrictAboutOutputDuringTests="true"
         beStrictAboutChangesToGlobalState="true"
         checkForUnintentionallyCoveredCode="false"
         printerClass="\Drupal\Tests\Listeners\HtmlOutputPrinter">

<php>
  <!-- Set error reporting to E_ALL. -->
  <ini name="error_reporting" value="32767"/>
  <!-- Do not limit the amount of memory tests take to run. -->
  <ini name="memory_limit" value="-1"/>
  <!-- Example SIMPLETEST_BASE_URL value: http://localhost -->
  <env name="SIMPLETEST_BASE_URL" value="http://drupal-8.localhost"/>
  <!-- Example SIMPLETEST_DB value: mysql://username:password@localhost/databasename#table_prefix -->
  <env name="SIMPLETEST_DB" value="mysql://drupal-8:drupal-8@localhost/drupal-8"/>
  <!-- Example BROWSERTEST_OUTPUT_DIRECTORY value: /path/to/webroot/sites/simpletest/browser_output -->
  <env name="BROWSERTEST_OUTPUT_DIRECTORY" value="/var/www/sites/default/files/simpletest"/>
</php> 
</phpunit>

Permission problems

Note: functional tests have to be invoked with a user in the same group as the
web server user. You can either configure Apache (or nginx) to run as your own
system user or run tests as a privileged user instead.

To develop locally, a straightforward - but also less secure - approach is to
run tests as your own system user. To achieve that, change the default Apache
user to run as your system user. Typically, you'd need to modify
`/etc/apache2/envvars` on Linux or `/etc/apache2/httpd.conf` on Mac.

Example for Linux:

export APACHE_RUN_USER=<your-user>
export APACHE_RUN_GROUP=<your-group>

Example for Mac:

User your-user
Group your-group

If the default user is e.g. `www-data`, the above functional tests will have to
be invoked with sudo instead:

export SIMPLETEST_DB='mysql://root@localhost/dev_d8'
export SIMPLETEST_BASE_URL='http://d8.dev'
sudo -u www-data -E ./vendor/bin/phpunit -c core --testsuite functional
sudo -u www-data -E ./vendor/bin/phpunit -c core --testsuite functional-javascript

Run one specific test

Simply specify the file name, example;

cd core
../vendor/bin/phpunit tests/Drupal/Tests/Core/Password/PasswordHashingTest.php 

List available groups:

To facilitate running specific tests, test authors use annotations to place their tests in one or more groups.
../vendor/bin/phpunit --list-groups

Run one specific group of tests:

../vendor/bin/phpunit --group Groupname

Run multiple groups of tests:

../vendor/bin/phpunit --group Group1,Group2

Exclude tests:
../vendor/bin/phpunit --exclude-group Groupname

Run a specific method:
../vendor/bin/phpunit --filter=MyMethodTest

Generate a code coverage report:
../vendor/bin/phpunit --coverage-html /tmp/report
And then open /tmp/report/index.html in your browser to review.

For a complete discussion of command line options when running tests, see PHPUnit's The Command-Line Test Runner

Run All PHPUnit Tests The Way The Testbot Does It

Running tests as described above is quick and easy, but it can be helpful to see how the drupal.org testbot runs your tests. This takes considerably longer, but helps you understand things from the testbot's point of view.

The first step is to make sure you have a full working Drupal installation, with the Testing module enabled.

Then, from the command line you can type:

php core/scripts/run-tests.sh PHPUnit

This specifies to the run-tests.sh script that it should run the PHPUnit group of tests. This group is internally generated by the script and its integration with SimpleTest, and represents any test that inherits \PHPUnit_Framework_TestCase including Drupal\Tests\UnitTestCase.

What to do with skipped tests

Skipped tests are usually an indication that your test environment is missing something, often a database connection. You can get more information about the skipped test by adding the -v parameter to your phpunit command.

If you get an error similar to:

InvalidArgumentException: There is no database connection so no tests can be run. You must provide a SIMPLETEST_DB environment variable, like "sqlite://localhost//tmp/test.sqlite", to run PHPUnit based functional tests outside of run-tests.sh. 

That means you have not set up your local phpunit.xml file copy correctly, see the kernel tests and browser tests section above.

Tests found when calling PHPUnit directly, but no tests found by drupal.org's testbot

The Drupal test runner (core/scripts/run-tests.sh) discovers tests in a different way than running phpunit directly. Confirm that your test conforms to the Drupal test runner standard as documented in PHPUnit file structure, namespace, and required metadata: Contributed Modules.

Most likely the Drupal test runner is not able to find your class in the autoloader, which means that the namespace and directory do not conform to the PSR-4 standard.

Tests pass locally but fail when run by drupal.org's testbot

One possible explanation for this is that you are introducing a new dependency to your module and testbot is not yet aware of this. If this is the case consider adding a test_dependencies property to your mymodule.info.yml file and committing it immediately. After pushing this change to drupal.org it can take up to 24 hours for testbot to become aware of your new dependency.

Another reason for locally-passing tests failing on the testbot is that locally you may be using a later 'dev' version of a 3rd-party dependency module, but the Drupal testbot is using only the most recent tagged release. If code changes have been made in the dev release which are necessary for your tests to pass, these will not be available to testbot in the official tagged release of the dependency module, and could be the cause of the failures.

Have another possible explanation for this mismatch? Add it here...

Further resources