Follow-up for #2232861: Create BrowserTestBase for web-testing on top of Mink

Problem/Motivation

The tests from this issue #2232861: Create BrowserTestBase for web-testing on top of Mink create a fail for some tests which do functional testing.

The failure is not related to the test, but the testing base class which requires an environment variable: SIMPLETEST_BASE_URL

Since it is an environmental concern and not a pass/fail for a test, the test should not fail.

One main reason this is a problem is that I can't run PHPUnit from the command line without a failure, unless I supply this environmental variable. This violates the isolation of the tests.

$ ./vendor/bin/phpunit --group simpletest
PHPUnit 4.4.2 by Sebastian Bergmann.

Configuration read from /Users/paulmitchum/projects/drupal8/core/phpunit.xml.dist

...............................................................  63 / 147 ( 42%)
............................................................... 126 / 147 ( 85%)
...................EE

Time: 6.71 seconds, Memory: 117.50Mb

There were 2 errors:

1) Drupal\Tests\simpletest\Functional\BrowserTestBaseTest::testGoTo
InvalidArgumentException: You must provide a SIMPLETEST_BASE_URL environment variable to run PHPUnit based functional tests.

/Users/paulmitchum/projects/drupal8/core/modules/simpletest/src/BrowserTestBase.php:242

2) Drupal\Tests\simpletest\Functional\BrowserTestBaseTest::testForm
InvalidArgumentException: You must provide a SIMPLETEST_BASE_URL environment variable to run PHPUnit based functional tests.

/Users/paulmitchum/projects/drupal8/core/modules/simpletest/src/BrowserTestBase.php:242
                                       
FAILURES!                              
Tests: 147, Assertions: 213, Errors: 2.

Proposed resolution

\Drupal\simpletest\BrowserTestBase should skip tests if the environment variable isn't available, if it's really a hard requirement.

Further development (if it's not really a hard requirement) could include refactoring \Drupal\simpletest\BrowserTestBase to create a useful base url if the environment variable isn't present.

Remaining tasks

A possible follow-up would be to not use BrowserTestBase to regression-test BrowserTestBase, but that's out of scope here.

User interface changes

API changes

CommentFileSizeAuthor
#9 2478247_9.patch622 byteshkirsman
#1 2478247_1.patch739 bytesmile23

Issue fork drupal-2478247

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

mile23’s picture

mile23
Primary language English
Location Seattle, WA
commented
Status: Active » Needs review
StatusFileSize
new 2478247_1.patch739 bytes

This patch marks any test as skipped if the environment variable isn't present.

$ ./vendor/bin/phpunit --group simpletest
PHPUnit 4.4.2 by Sebastian Bergmann.

Configuration read from /Users/paulmitchum/projects/drupal8/core/phpunit.xml.dist

...............................................................  63 / 147 ( 42%)
............................................................... 126 / 147 ( 85%)
...................SS

Time: 6.75 seconds, Memory: 117.50Mb

OK, but incomplete, skipped, or risky tests!
Tests: 147, Assertions: 213, Skipped: 2.
mile23’s picture

mile23
Primary language English
Location Seattle, WA
commented
Issue summary: View changes
larowlan’s picture

larowlan
Location 🇦🇺🏝.au GMT+10
commented
Status: Needs review » Reviewed & tested by the community

Thanks

wim leers’s picture

wim leers
Location Ghent 🇧🇪🇪🇺
commented
Issue tags: +DX (Developer Experience)

Yes, please!

alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
commented
Status: Reviewed & tested by the community » Fixed

Nice. Just to note you can do --testsuite unit to skip functional testing too - but +1 to this as well.

Committed 5b55ea8 and pushed to 8.0.x. Thanks!

  • alexpott committed 5b55ea8 on 8.0.x 
    Issue #2478247 by Mile23: SIMPLETEST_BASE_URL is an environmental...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

Alexander Kalashnik made their first commit to this issue’s fork.

hkirsman’s picture

hkirsman commented
StatusFileSize
new 2478247_9.patch622 bytes

I see it has been removed. -testsuite unit does not work - it'll fail before running.

Need this is in our docker environment and Teamcity where there isn't web server running.