Problem/Motivation

When the new \Drupal\KernelTests\KernelTestBase is used as base for a test, that test fails with:

InvalidArgumentException: Required prefix configuration is missing

This is because FileCache is not properly initialized.

Proposed resolution

Initialize FileCache before tests are running. release the memory cache when finishing.

Remaining tasks

None.

User interface changes

None.

API changes

New public static method \Drupal\Component\FileCache/FileCache::reset() tat releases the memory.

Data model changes

None.

Beta phase evaluation

Reference: https://www.drupal.org/core/beta-changes
Issue category Bug because it makes tests based on \Drupal\KernelTests\KernelTestBase to fail.
Issue priority Critical \Drupal\KernelTests\KernelTestBase cannot be used.
Disruption No disruption.

Comments

claudiu.cristea created an issue. See original summary.

dawehner’s picture

What happens if you run this test via phpunit itself, which is the recommended way?

claudiu.cristea’s picture

Well, my real case needs access to db and files but I got this error while building the test from scratch. I wrote this fake test only to reveal this failure.

claudiu.cristea’s picture

Here's with phpunit:

$ ./vendor/bin/phpunit --filter="Foo"
PHPUnit 4.6.4 by Sebastian Bergmann and contributors.

Configuration read from /Users/clau/development/8/ajax/core/phpunit.xml.dist

E

Time: 2.07 seconds, Memory: 66.00Mb

There was 1 error:

1) Drupal\Tests\system\Kernel\Extension\FooTest::testBar
InvalidArgumentException: There is no database connection so no tests can be run. You must provide a SIMPLETEST_DB environment variable, like "sqlite://localhost//tmp/test.sqlite", to run PHPUnit based functional tests outside of run-tests.sh.

/Users/clau/development/8/ajax/core/tests/Drupal/KernelTests/KernelTestBase.php:404
/Users/clau/development/8/ajax/core/tests/Drupal/KernelTests/KernelTestBase.php:282
/Users/clau/development/8/ajax/core/tests/Drupal/KernelTests/KernelTestBase.php:214

FAILURES!
Tests: 1, Assertions: 0, Errors: 1.
claudiu.cristea’s picture

Setting the env var SIMPLETEST_DB to a valid connection string fixes the problem when running from command line. But it's still broken in UI.

claudiu.cristea’s picture

Now I'm getting another problem when adding modules.

namespace Drupal\Tests\system\Kernel\Extension;
use Drupal\KernelTests\KernelTestBase;

/**
 * @group Extension
 */
class FooTest extends KernelTestBase {
  public static $modules = ['system'];

  public function testBar() {
    $this->assertTrue(TRUE);
  }
}

this produces

$ ./vendor/bin/phpunit --filter="FooTest"
PHPUnit 4.6.4 by Sebastian Bergmann and contributors.

Configuration read from /Users/clau/development/8/ajax/core/phpunit.xml.dist

E

Time: 2.11 seconds, Memory: 66.00Mb

There was 1 error:

1) Drupal\Tests\system\Kernel\Extension\FooTest::testBar
InvalidArgumentException: Required prefix configuration is missing

/Users/clau/development/8/ajax/core/lib/Drupal/Component/FileCache/FileCache.php:59
/Users/clau/development/8/ajax/core/lib/Drupal/Component/FileCache/FileCacheFactory.php:61
/Users/clau/development/8/ajax/core/lib/Drupal/Core/Extension/ExtensionDiscovery.php:101
/Users/clau/development/8/ajax/core/tests/Drupal/KernelTests/KernelTestBase.php:498
/Users/clau/development/8/ajax/core/tests/Drupal/KernelTests/KernelTestBase.php:334
/Users/clau/development/8/ajax/core/tests/Drupal/KernelTests/KernelTestBase.php:215

FAILURES!
Tests: 1, Assertions: 0, Errors: 1.

dawehner’s picture

Status: Active » Needs review
StatusFileSize
new2.01 KB

Maybe something like this?

claudiu.cristea’s picture

Manually tested. This is fixing "Required prefix configuration is missing" error.

jibran’s picture

I don't think we can really test this. I applied this locally and it fixes the issue for me.

+++ b/core/tests/Drupal/KernelTests/KernelTestBase.php
@@ -478,6 +479,29 @@ private function getCompiledContainerBuilder(array $modules) {
+  protected function initFileCache() {

Function doc block is missing.

claudiu.cristea’s picture

StatusFileSize
new2.03 KB
new945 bytes

I tried also to add a test for that but no luck.

Function doc block is missing.

I guess the only thing needed here is to move that comment to a doc block.

jibran’s picture

Status: Needs review » Reviewed & tested by the community

Thanks for the fix. It's RTBC.

alexpott’s picture

Status: Reviewed & tested by the community » Postponed

Before we add any more KTBTNG's we need to fix https://www.drupal.org/node/2553533 - each new test added increases the chance of random fail due to test id reuse.

neclimdul’s picture

Status: Postponed » Needs review

#2553533: KernelTestBaseTNG™ is not cleaning up after itself is in and did a once over and this looks mostly ok with me and it does fix it but not sure about this.

+++ b/core/tests/Drupal/KernelTests/KernelTestBase.php
@@ -479,6 +480,32 @@ private function getCompiledContainerBuilder(array $modules) {
+    FileCacheFactory::setConfiguration($configuration);
+    FileCacheFactory::setPrefix(Settings::getApcuPrefix('file_cache', $this->root));

@@ -494,6 +521,8 @@ private function getCompiledContainerBuilder(array $modules) {
+    $this->initFileCache();
+

Do we want to reset these values each time we get a list of extensions? Currently the 2 places we call it we do need to initialize filecache but I'm not sure that is an appropriate assumption to build into the method.

dawehner’s picture

Do we want to reset these values each time we get a list of extensions? Currently the 2 places we call it we do need to initialize filecache but I'm not sure that is an appropriate assumption to build into the method.

I think yeah we should avoid any sideeffects if possible. Pratically it doesn't matter at the moment because we use process isolation but still ... once we might get rid of it, its better to avoid it.

neclimdul’s picture

Well my point was its not a clear side effect of the function. We are in fact introducing a side effect by changing the value.

geertvd’s picture

StatusFileSize
new2.02 KB

I noticed this needed a reroll while working on #2556855: Port ViewKernelTestBase to extend from KernelTestBaseTNG™

dawehner’s picture

Status: Needs review » Needs work

We should reset in tearDown() ...

claudiu.cristea’s picture

@dawehner, I see no reset/wipe method on FileCache or FileCacheFactory. Or should we use somehow a __destruct() method?

dawehner’s picture

Well, then FileCache should get one ...

claudiu.cristea’s picture

Status: Needs work » Needs review
StatusFileSize
new4.7 KB
new2.92 KB

Not sure I got the idea :)

Status: Needs review » Needs work

The last submitted patch, 20: 2553661-20.patch, failed testing.

dawehner’s picture

Well actually I thought about a reset() method which just resets \Drupal\Component\FileCache\FileCache::$cached

geertvd’s picture

Status: Needs work » Needs review
StatusFileSize
new1.4 KB
new3.18 KB

Simply this then. Interdiff is from #16

Status: Needs review » Needs work

The last submitted patch, 23: 2553661-23.patch, failed testing.

geertvd’s picture

StatusFileSize
new3.66 KB
new1.16 KB
geertvd’s picture

Status: Needs work » Needs review
neclimdul’s picture

+++ b/core/tests/Drupal/KernelTests/KernelTestBase.php
@@ -495,6 +523,8 @@ private function getCompiledContainerBuilder(array $modules) {
+    $this->initFileCache();
+

Seems like we could move this to setUp() to match the reset in teardown. Not tied to this since dawehner seems to like it where it is but it still makes sense to me. Other then that, if a FileCache maintainer signs of on the new method I think we're good.

dawehner’s picture

  1. +++ b/core/lib/Drupal/Component/FileCache/FileCacheInterface.php
    @@ -61,4 +61,9 @@ public function set($filepath, $data);
    +  /**
    +   * Resets the static cache.
    +   */
    +  public static function reset();
    

    IMHO you don't need to add a static function to an interface.

  2. +++ b/core/tests/Drupal/KernelTests/KernelTestBase.php
    @@ -495,6 +523,8 @@ private function getCompiledContainerBuilder(array $modules) {
       private function getExtensionsForModules(array $modules) {
    +    $this->initFileCache();
    
    @@ -625,6 +655,9 @@ protected function tearDown() {
    +    // Clean FileCache cache.
    +    FileCache::reset();
    +
    

    I like the idea to move it to setup() itself.

xano’s picture

Title: Very simple KernelTestBaseTNG™ failing with exception » KernelTestBase fails to set up FileCache
StatusFileSize
new3.17 KB
new2.6 KB

I also replaced string FQNs with ::class usages.

neclimdul’s picture

Status: Needs review » Reviewed & tested by the community

WFM. Thanks for putting up with my nit-picks guys.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 29: drupal_2553661_29.patch, failed testing.

Status: Needs work » Needs review

geertvd queued 29: drupal_2553661_29.patch for re-testing.

geertvd’s picture

Status: Needs review » Reviewed & tested by the community

Seems like a random fail, setting back to rtbc

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 29: drupal_2553661_29.patch, failed testing.

geertvd’s picture

These are different fails then the ones before, so i'm retesting this again.

Status: Needs work » Needs review

geertvd queued 29: drupal_2553661_29.patch for re-testing.

Status: Needs review » Needs work

The last submitted patch, 29: drupal_2553661_29.patch, failed testing.

geertvd’s picture

Not sure why this is failing, EntityFileTest extends KernelTestBase but not KernelTestBaseTNG

borisson_’s picture

The test failures of EntityFileTest are the same here as they are in in #2358319: Alt tag missing on user images, so I don't think these are related to this patch.

dawehner’s picture

geertvd’s picture

As mentioned in some other tickets, there was also an issue with bot 3128 which was running these failing tests. Since it's been taking out of rotation it should pass again now.

Status: Needs work » Needs review

geertvd queued 29: drupal_2553661_29.patch for re-testing.

geertvd’s picture

Status: Needs review » Reviewed & tested by the community

Setting back to RTBC as per #30

alexpott’s picture

Status: Reviewed & tested by the community » Needs review
+++ b/core/lib/Drupal/Component/FileCache/FileCache.php
@@ -153,4 +153,11 @@ public function delete($filepath) {
+  /**
+   * Resets the static cache.
+   */
+  public static function reset() {
+    static::$cached = [];
+  }

Why do we need the reset? Each KernelTestBase is run inside it's own process.

dawehner’s picture

Why do we need the reset? Each KernelTestBase is run inside it's own process.

I hope we don't assume that.

alexpott’s picture

@dawehner of course we assume that - in order to run KernelTestBaseTNG in the same process we have to remove all procedural code from Drupal.

neclimdul’s picture

I guess the point is, every time we build that assumption in we add one more hurdle to not having that assumption. Which isn't to say we are going to remove all the procedural code from Drupal, just that at some point we might have tests opt in to running in a separate process rather then that being the default or something like that. It seems like cleaning up is just the right thing to do.

xano’s picture

Agreed with @dawehner and @neclimdul.

I am curious though as to why different FileCache instances share the same static property for caching data.

dawehner’s picture

@dawehner of course we assume that - in order to run KernelTestBaseTNG in the same process we have to remove all procedural code from Drupal.

But we should try to remove the surface area of problems as much as possible. If we know about this static cache, we should clear it.

neclimdul’s picture

Status: Needs review » Reviewed & tested by the community

I think concerns have been addressed so going to put this back RTBC.

claudiu.cristea’s picture

Issue summary: View changes
Status: Reviewed & tested by the community » Needs review
StatusFileSize
new2.6 KB
new602 bytes

Nit: We are initializing the FileCache, not the FileCacheFactory.

Added Issue Summary and Beta Evaluation.

claudiu.cristea’s picture

Issue summary: View changes
neclimdul’s picture

Status: Needs review » Reviewed & tested by the community

That's fair. We are just using the factory to do it. Better docs, still RTBC.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 51: 2553661-51.patch, failed testing.

neclimdul queued 51: 2553661-51.patch for re-testing.

neclimdul’s picture

The last submitted patch, 51: 2553661-51.patch, failed testing.

claudiu.cristea queued 51: 2553661-51.patch for re-testing.

claudiu.cristea’s picture

Status: Needs work » Needs review
dawehner’s picture

Status: Needs review » Reviewed & tested by the community

Let it be green while its green.

alexpott’s picture

Status: Reviewed & tested by the community » Needs work

So this is completely lacking tests which is a shame. All we need to do is change \Drupal\system\Tests\Extension\ModuleHandlerTest to install system and change testModuleList() to start with the system module in the list.

And I'm still not a fan of the reset. It solves the issue in one place whereas we actually have more instances of this. What we need is a drupal_static() for classes - there is an issue for this somewhere. Can an @todo be added to remove this once that is done.

dawehner’s picture

Status: Needs work » Needs review
StatusFileSize
new4.58 KB
new2.4 KB

There we go

Status: Needs review » Needs work

The last submitted patch, 62: 2553661-62.patch, failed testing.

neclimdul’s picture

neclimdul queued 62: 2553661-62.patch for re-testing.

neclimdul’s picture

Status: Needs work » Reviewed & tested by the community

like it.

alexpott’s picture

Status: Reviewed & tested by the community » Fixed

Committed 0f8b5b1 and pushed to 8.0.x. Thanks!

  • alexpott committed 0f8b5b1 on 8.0.x
    Issue #2553661 by claudiu.cristea, geertvd, dawehner, Xano, neclimdul:...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.