Problem/Motivation

While working on https://github.com/jsdrupal/nightwatch we realized we could autostart chrome/chromdriver, with default settings, so people don't have to really setup stuff to run these tests.

Proposed resolution

Use a similar concept for the PHP based tests, too.

Remaining tasks

User interface changes

API changes

Data model changes

Comments

dawehner created an issue. See original summary.

dawehner’s picture

Issue tags: +PHPUnit
alexpott’s picture

Status: Active » Needs review
StatusFileSize
new9.52 KB

Got a browser test running with zero config. Now we just need to implement starting up chromedriver. There's a problem with PHPUnit and phpunit.xml env var declarations taking precedence of the things already set up. This behaviour is fixed in PHPUnit 6 but this means we probably need to read that too to provide a consistent behaviour for the user.

Status: Needs review » Needs work

The last submitted patch, 4: 2946472-4.patch, failed testing. View results

alexpott’s picture

Status: Needs work » Needs review
StatusFileSize
new9.53 KB

Downgrading symfony/dotenv

dawehner’s picture

Right now the chromedriver on JS is installed using the package.json. I'm wondering whether we could leverage that somehow too.

alexpott’s picture

StatusFileSize
new10.89 KB

Not sure about #7 but here's a zero config js test working if chromedriver is installed in the path. So I guess all we would need to do is add wherever package.json puts chromedriver to the paths to search.

alexpott’s picture

StatusFileSize
new11.71 KB

Now supporting the most common PHPUnit argument / options. Should be able to run any PHPUnit test we have bar the Phantom ones and that is solvable too.

alexpott’s picture

StatusFileSize
new675 bytes
new11.71 KB

Whoops - only filter was working.

So php ./core/scripts/php-test.php run-phpunit --testsuite unit works as expected. Obviously that's kinda silly because we don't need to use this to run unit tests but the it proves it works and all the browser tests would take ages.

alexpott’s picture

StatusFileSize
new3.28 KB
new12.8 KB

More docs.

dawehner’s picture

  1. +++ b/core/.env.example
    @@ -0,0 +1,29 @@
    +# This is a dotenv file used by JavaScript tasks.
    

    I'm curious, should we adapt some of this documentation to point more towards phpunit based testing as well?

  2. +++ b/core/.gitignore
    @@ -9,3 +9,5 @@ phpunit.xml
    diff --git a/core/composer.json b/core/composer.json
    
    diff --git a/core/composer.json b/core/composer.json
    index 62e765947d..8f575e94e9 100644
    
    index 62e765947d..8f575e94e9 100644
    --- a/core/composer.json
    
    --- a/core/composer.json
    +++ b/core/composer.json
    
    +++ b/core/composer.json
    +++ b/core/composer.json
    @@ -62,7 +62,8 @@
    
    @@ -62,7 +62,8 @@
             "phpspec/prophecy": "^1.4",
             "symfony/css-selector": "^3.4.0",
             "symfony/phpunit-bridge": "^3.4.3",
    -        "symfony/debug": "^3.4.0"
    +        "symfony/debug": "^3.4.0",
    +        "symfony/dotenv": "^3.4.0"
         },
         "replace": {
             "drupal/action": "self.version",
    

    Can we have a composer script for this?

  3. +++ b/core/scripts/php-test.php
    @@ -0,0 +1,22 @@
    +/**
    + * @file
    + * A command line application to install drupal for tests.
    + */
    +
    +use Drupal\PhpTestRunner\PhpTestRunnerApplication;
    

    I'm curious whether we could have the same entry point for all tests, basically run-tests.sh 2.0?

  4. +++ b/core/scripts/php-test.php
    @@ -0,0 +1,22 @@
    +$app = new PhpTestRunnerApplication('test-site', '0.1.0');
    

    Note: We should update the name

  5. +++ b/core/tests/Drupal/PhpTestRunner/Commands/RunPhpUnitCommand.php
    @@ -0,0 +1,170 @@
    +/**
    + * Command to run Drupal's PHPUnit tests.
    + *
    + * This starts up a webserver and chromedriver as required ensuring tests can
    + * be run with zero configuration.
    + *
    + * @internal
    + */
    +class RunPhpUnitCommand extends Command {
    

    We should document that for more advanced needs you might should run it directly still. One more advanced usage would be to actually run the entire test suite.

  6. +++ b/core/tests/Drupal/PhpTestRunner/Commands/RunPhpUnitCommand.php
    @@ -0,0 +1,170 @@
    +  protected $webserver;
    ...
    +  protected $chromedriver;
    

    What about using $webserverProcess and $chromedriverProcess as name?

  7. +++ b/core/tests/Drupal/PhpTestRunner/Commands/RunPhpUnitCommand.php
    @@ -0,0 +1,170 @@
    +      ->setHelp('@todo')
    

    I think this help text should point to Phpunit documentation

  8. +++ b/core/tests/Drupal/PhpTestRunner/Commands/RunPhpUnitCommand.php
    @@ -0,0 +1,170 @@
    +    $tests = implode(' ', $input->getArgument('test'));
    +    $command = "./vendor/bin/phpunit --verbose -c ./core $tests";
    

    It is nice that you support multiple arguments.

  9. +++ b/core/tests/Drupal/PhpTestRunner/Commands/RunPhpUnitCommand.php
    @@ -0,0 +1,170 @@
    +    if ($web_user = getenv('DRUPAL_TEST_WEBSERVER_USER')) {
    +      $command = "sudo -u $web_user $command";
    +    }
    

    What a nice idea ;)

  10. +++ b/core/tests/Drupal/PhpTestRunner/PhpTestRunnerApplication.php
    @@ -0,0 +1,28 @@
    +class PhpTestRunnerApplication extends Application {
    +
    +  /**
    +   * {@inheritdoc}
    +   */
    +  protected function getDefaultCommands() {
    +    // Even though this is a single command, keep the HelpCommand (--help).
    +    $default_commands = parent::getDefaultCommands();
    +    $default_commands[] = new RunPhpUnitCommand();
    +    return $default_commands;
    +  }
    +
    

    So if we know there will be just one command anyway, could we skip having a custom class and wire it up in the main entry point?

justafish’s picture

StatusFileSize
new674 bytes
justafish’s picture

StatusFileSize
new14.46 KB

🤦‍♀️

justafish’s picture

StatusFileSize
new16.93 KB
new2.46 KB
alexpott’s picture

StatusFileSize
new7.79 KB
new13.34 KB

Patch addresses #12

Re #12.3 - I think there is value using PHP to run PHP tests and JS to run JS tests. Not sure we want a single runner. It also makes it simple just to pass things through to PHPUnit. This doesn't attempt to pass things to run-tests.sh to run Simpletest tests for example.

alexpott’s picture

I was also thinking we could implement a phpunit-other-args option to pass more options on to phpunit if people want.

dawehner’s picture

I was also thinking we could implement a phpunit-other-args option to pass more options on to phpunit if people want.

Let's open up a followup for that?

Re #12.3 - I think there is value using PHP to run PHP tests and JS to run JS tests. Not sure we want a single runner. It also makes it simple just to pass things through to PHPUnit. This doesn't attempt to pass things to run-tests.sh to run Simpletest tests for example.

Fair point. To be honest having composer run-script tests and npm run tests would be enough of consistency for me.

+++ b/core/tests/Drupal/PhpTestRunner/PhpTestRunnerCommand.php
@@ -0,0 +1,209 @@
+    $env = $this->handleEnvironment($input);
+
+    if ($this->webserverProcess) {
+      $output->writeln('<info>Starting PHP webserver</info>');
+      $this->webserverProcess->start();
+    }
+    if ($this->chromedriverProcess) {
+      $output->writeln('<info>Starting Chromedriver</info>');
+      $this->chromedriverProcess->start();
+    }

I'm wondering: Should we check somehow whether we run unit tests and in that case not try to start up a webserver/chromedriver, given that both might be quite slow?

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

smustgrave’s picture

Status: Needs review » Needs work

If still needed. A follow up ticket was requested in #18

nod_’s picture

Patch or MR doesn't apply anymore
The last patch or MR doesn't apply to the target branch, please reroll the code so that it can be reviewed by the automated testbot.

D10 version needed
At this time we would need a D10.1.x patch or MR for this issue.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

andypost’s picture

andypost’s picture

Issue tags: +Needs reroll
Related issues: +#3387737: Split PHP image into php(cli/apache) and yarn(node/nightwatch)

As selenium 4 is out and W3C Webdriver standardized it will need to prepare specific docker image for new CI.

Faced while worked on split of PHP image #3387737: Split PHP image into php(cli/apache) and yarn(node/nightwatch)

vsujeetkumar’s picture

StatusFileSize
new12.61 KB
new8.67 KB

Re-roll patch created for 11.x.

vsujeetkumar’s picture

Status: Needs work » Needs review
StatusFileSize
new12.62 KB
new819 bytes

Fixed the CCF Issue. Please have a look.

smustgrave’s picture

Status: Needs review » Needs work

Can the follow in #18 be addressed if it's needed or not?

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.