Testing topic documentation is way out of date

Yes please! Ditto core/tests/README.md which is also wildly stale, especially wrt FunctionalJavascript tests.

https://www.drupal.org/docs/8/phpunit/running-phpunit-tests fails to mention you have to modify phpunit.xml to get FunctionalJavascript tests to work, and it talks about PhantomJS (which, AFAIK, is no longer needed/recommended/used). If you forget, they all silently pass, even if you have $this->assertTrue(FALSE). That should probably be a separate bug report. Maybe that's a run-tests.sh bug.

https://www.drupal.org/docs/8/phpunit/phpunit-javascript-testing-tutorial seems reasonably up-to-date, but core/tests/README.md is woefully behind the times.

Another evil pain point: if you configure your phpunit.xml to get the JS tests to work, everything else fails along these lines:

    Behat\Mink\Exception\UnsupportedDriverActionException: Response headers are not available from Drupal\FunctionalJavascriptTests\DrupalSelenium2Driver

Argh. I sure wish there was a simple way to actually configure a core checkout to run all the tests locally. ;)

Ahh, great. The run-tests.sh bug from the first part of #4 is here: #2905007: Allow run-tests.sh to report skipped/incomplete PHPUnit tests. Glad that's already known. ;)

Re: #4 "Another evil pain point...", turns out that defining <env name="MINK_DRIVER_CLASS" value="Drupal\FunctionalJavascriptTests\DrupalSelenium2Driver"/> is totally unnecessary and causes trouble. I removed it from that handbook page.

From Slack:

alexpott: @dww One day we should totally re-organise these docs
alexpott: Top level: Types of tests, Running tests, Writing tests
alexpott: Inside running tests - separate pages for each type and a page on phpunit.xml configs
alexpott: inside writing tests - section per test type

I couldn't agree more. ;) Pasting here for reference.

Note: he's actively in the middle of that rearrangement as I type this...

FYI: @alexpott and I have done a fair bit of review, re-org, re-testing, and edits to the PHPUnit in Drupal 8 guide. In particular, these pages are now pretty much right:

https://www.drupal.org/docs/8/phpunit/running-phpunit-tests
https://www.drupal.org/docs/8/phpunit/running-phpunit-javascript-tests
https://www.drupal.org/docs/8/phpunit/phpunit-javascript-test-writing-tu...

really nice! Couple of things:

1. +++ b/core/core.api.php
   @@ -1062,7 +1062,7 @@
   + * Overview of PHPUnit automated tests
   

   missing period at the end

2. +++ b/core/core.api.php
   @@ -1083,123 +1083,60 @@
   + * start with 'test' and have no arguments; for example, testYourTestCase().
   

   This is true for Simpletest but not for PHPUnit tests, which can use @dataProvider

3. +++ b/core/tests/README.md
   @@ -1,140 +1,78 @@
   +* PHP 7.1 or higher
   

   is this really needed? not that I know.

+++ b/core/tests/README.md
@@ -1,140 +1,78 @@
-## Nightwatch tests
-
-- Ensure your vendor directory is populated (e.g. by running `composer install`)

-- Ensure your vendor directory is populated (e.g. by running `composer install`)
-- If you're running PHP 7.0 or greater you will need to upgrade PHPUnit with `composer run-script drupal-phpunit-upgrade`

-- If you're running PHP 7.0 or greater you will need to upgrade PHPUnit with `composer run-script drupal-phpunit-upgrade`
-- Install [Node.js](https://nodejs.org/en/download/) and [yarn](https://yarnpkg.com/en/docs/install). The versions required are specificed inside core/package.json in the `engines` field

-- Install [Node.js](https://nodejs.org/en/download/) and [yarn](https://yarnpkg.com/en/docs/install). The versions required are specificed inside core/package.json in the `engines` field
-- Install [Google Chrome](https://www.google.com/chrome/browser/desktop/index.html)

-- Install [Google Chrome](https://www.google.com/chrome/browser/desktop/index.html)
-- Inside the `core` folder, run `yarn install`

-- Inside the `core` folder, run `yarn install`
-- Configure the nightwatch settings by copying `.env.example` to `.env` and editing as necessary.

-- Configure the nightwatch settings by copying `.env.example` to `.env` and editing as necessary.
-- Ensure you have a web server running (as instructed in `.env`)

-- Ensure you have a web server running (as instructed in `.env`)
-- Again inside the `core` folder, run `yarn test:nightwatch` to run the tests. By default this will output reports to `core/reports`

-- Again inside the `core` folder, run `yarn test:nightwatch` to run the tests. By default this will output reports to `core/reports`
-- Nightwatch will run tests for core, as well as contrib and custom modules and themes. It will search for tests located under folders with the pattern `**/tests/**/Nightwatch/(Tests|Commands|Assertions)`

-- Nightwatch will run tests for core, as well as contrib and custom modules and themes. It will search for tests located under folders with the pattern `**/tests/**/Nightwatch/(Tests|Commands|Assertions)`
-- To run only core tests, run `yarn test:nightwatch --tag core`

-- To run only core tests, run `yarn test:nightwatch --tag core`
-- To skip running core tests, run `yarn test:nightwatch --skiptags core`

-- To skip running core tests, run `yarn test:nightwatch --skiptags core`
-- To run a single test, run e.g. `yarn test:nightwatch tests/Drupal/Nightwatch/Tests/exampleTest.js`

-- To run a single test, run e.g. `yarn test:nightwatch tests/Drupal/Nightwatch/Tests/exampleTest.js`
-
-Nightwatch tests can be placed in any folder with the pattern `**/tests/**/Nightwatch/(Tests|Commands|Assertions)`. For example:
-```
-tests/Nightwatch/Tests
-src/tests/Nightwatch/Tests
-tests/src/Nightwatch/Tests
-tests/Nightwatch/Commands
-```
-
-It's helpful to follow existing patterns for test placement, so for the action module they would go in `core/modules/action/tests/src/Nightwatch`.
-The Nightwatch configuration, as well as global tests, commands, and assertions which span many modules/systems, are located in `core/tests/Drupal/Nightwatch`.
-
-If your core directory is located in a subfolder (e.g. `docroot`), then you can edit the search directory in `.env` to pick up tests outside of your Drupal directory.
-Tests outside of the `core` folder will run in the version of node you have installed. If you want to transpile with babel (e.g. to use `import` statements) outside of core,
-then add your own babel config to the root of your project. For example, if core is located under `docroot/core`, then you could run `yarn add babel-preset-env` inside
-`docroot`, then copy the babel settings from `docroot/core/package.json` into `docroot/package.json`.

any reason why the entire nightwatch section is removed?

1. +++ b/core/tests/README.md
   @@ -1,140 +1,78 @@
   +You need the Drupal core development depedendencies installed, in order to run
   

   spelling: dependencies

2. +++ b/core/tests/README.md
   @@ -1,140 +1,78 @@
   +### Debug output location
    
   -Example for Linux:
   +Create a directory called simpletest in sites/default and make sure that it it
   +is writable by the web server and/or all users.
   

   I'm not sure where this comes from but I think this doesn't belong here. I think it can be merged into the next section - specifically the bits about BROWSERTEST_OUTPUT_DIRECTORY (which I think this is trying to help you set up) and the printerClass thingy.

   Or maybe this is saying that in order to run tests you need a sites/simpletest that is writable by both the user running the tests and the webserver? If so that's not debug output. That's the site directory of the site under test which includes its file directory, twig cache, its settings.php, etc...

3. +++ b/core/tests/README.md
   @@ -1,140 +1,78 @@
   +To run tests, you will need to invoke this executable with a user in the same
   +group as the web server user, or with access to files owned by the web server
   +user. To maintain web server user access to files created by tests, make sure
   +that you have
   +`$settings['file_chmod_directory'] = 02775;`
   +in your settings.php or local.settings.php file.
   

   This is not necessarily true. It depends on how your file system is set up and works and how you run the tests. Not everyone needs to do this. I might put this in a troubleshooting section.

Alas. Oh well. #2949715 is the older issue, so this should be closed as the dup. We can merge efforts over there. I see you and @alexpott are already involved, so it's just me and @Lendude who have to move over.

Cheers,
-Derek