TestSuiteBaseTest cannot be executed as standalone test

Hello,

Thanks for the patch.

I have just tested it against the 8.2.x and it solved the issue. I encountered this issue when trying to configure automated tests in PhpStorm. (#2782221-4: Result summary Area plugin not displayed when there is no result.)

Unfortunately for me, now I have another issue:

Warning: mkdir(): Permission denied Drupal\Component\PhpStorage\FileStorage->createDirectory()() (Line: 165)

And I don't know what is the directory that is tried to be created. I follow https://www.drupal.org/node/2116263 and https://www.drupal.org/node/2288559

As I tested against 8.2.x, should I change the status into RTBC?

I'm wondering whether we should add core/tests/TestSuites to the autoloader inside the bootstrap.php

This is a bug fix of the testing system that helps prevent regressions of the testing system. So it should be in the 8.1.x branch. The patch applies in 8.2.x and 8.3.x as well.

The idea with leaving TestSuites in another namespace was that it was all about discovery and not about testing per se. But since we have to test the discovery process, we can just move it to where it makes sense to autoload.

This patch moves the test suite classes to \Drupal\TestSuites\*

The test stays where it is. This solves this issue and also #2781203: Move TestSuite/* into a dedicated namespace.

$ ./vendor/bin/phpunit -c core/ core/tests/Drupal/Tests/TestSuites/TestSuiteBaseTest.php 
PHPUnit 4.8.11 by Sebastian Bergmann and contributors.

..

Time: 258 ms, Memory: 5.75Mb

OK (2 tests, 4 assertions)

This looks like a really nice solution to the problem ...

With the customized phpunit.yml what would happen if we copied the classes back to their old location? Usually we try not to break test runs in patch releases unless it's necessary to fix a (non-testing) bug.

I moved TestSuiteBase here because we're trying to test it and because we also have #2781203: Move TestSuite/* into a dedicated namespace

It was probably a mistake to put them in weird places to begin with. I'd hate to put a bunch of require_once all over this system to allow the test to run.

Also:

Usually we try not to break test runs in patch releases unless it's necessary to fix a (non-testing) bug.

This won't break test runs for the testbot or for anyone doing core dev unless they've rigged their own phpunit.xml file as mentioned.

If all your CI script does is say phpunit -c core/ then you're golden. And as @klausi mentioned, we did this previously to add the suite classes to begin with. I'm pretty sure other than this issue and the duplicate, we haven't heard of any problems.

We could split this back out into two issues and get @klausi's test running using a require_once to load the class we're resting, and then address the locations of the files in #2781203: Move TestSuite/* into a dedicated namespace. In that case, the patch in #2 is the way to go.

OK I prefer patch #2 for this issue in 8.2.x, then a new issue for 8.3.x for the move.

Here's a re-upload of #2 just to keep it clear. Please give all credit to @klausi. :-)

And a manual test run:

$ ./vendor/bin/phpunit -c core core/tests/Drupal/Tests/TestSuites/TestSuiteBaseTest.php 
PHPUnit 4.8.11 by Sebastian Bergmann and contributors.

..

Time: 332 ms, Memory: 5.75Mb

OK (2 tests, 4 assertions)

Thanks. RTBC-ing that patch, but please open the follow-up, I think that'd be OK for a minor release (i.e. 8.3.x).

The follow-up is #2781203: Move TestSuite/* into a dedicated namespace which I just re-opened. :-)

I think @catch, @effulgentsia, @Cottser, @webchick, and I also agreed this can be rc eligible as a non-disruptive testing improvement. Thanks @Mile23.

Committed/pushed to 8.3.x, thanks!

Leaving RTBC against 8.2.x.

Er rc eligible, so cherry picked.