Problem/Motivation

This is a child issue of #3057420: [meta] How to deprecate Simpletest with minimal disruption

We want to allow for the removal of the simpletest module from core.

As part of that process, we try removing the simpletest module: #3075490-17: Move simpletest module to contrib And discover that Drupal\Core\Test\FunctionalTestSetupTrait::prepareSettings() both a) expects there to be a simpletest module, and b) adds a configuration to it:

    // Add the parent profile's search path to the child site's search paths.
    // @see \Drupal\Core\Extension\ExtensionDiscovery::getProfileDirectories()
    $settings['conf']['simpletest.settings']['parent_profile'] = (object) [
      'value' => $this->originalProfile,
      'required' => TRUE,
    ];

This config sneak is then consumed in many places as a way to signal a special case for testing. Here's one: ExtensionDiscovery::setProfileDirectoriesFromSettings().

You can find these usages by searching for 'simpletest.settings' in the codebase.

This comes from the deep dark past: #2171683: Remove all Simpletest overrides and rely on native multi-site functionality instead No clear explanation is given as to why the simpletest.settings config namespace is used. It seems as though the config is attached to a config object so that it's not present unless you get that config object.


We also see FunctionalTestSetupTrait adding services in the simpletest namespace.

    if ($this->strictConfigSchema) {
      // Add a listener to validate configuration schema on save.
      $yaml = new SymfonyYaml();
      $content = file_get_contents($directory . '/services.yml');
      $services = $yaml->parse($content);
      $services['services']['simpletest.config_schema_checker'] = [
        'class' => ConfigSchemaChecker::class,
        'arguments' => ['@config.typed', $this->getConfigSchemaExclusions()],
        'tags' => [['name' => 'event_subscriber']],
      ];
      file_put_contents($directory . '/services.yml', $yaml->dump($services));
    }

Fortunately ConfigSchemaChecker is not in a simpletest namespace.

It's unclear from testing whether this matters or not.

Proposed resolution

Figure out how to pass this special-case testing config to consumers. We might consider moving it to the system module's configuration, thus perpetuating the anti-pattern. Or we might fix the whole thing and make testing not be a configured special case, thus improving the testing integrity of Drupal core and the maintainability of the testing systems. But probably the former rather than the latter.

Patch.

Review.

Remaining tasks

Perhaps refactor the way this configuration is passed around within #3038513: Move drupal_generate_test_ua() into the test system or other follow-up.

User interface changes

API changes

Data model changes

Release notes snippet

Comments

Mile23 created an issue. See original summary.

mile23’s picture

Issue summary: View changes
mile23’s picture

Issue summary: View changes
mile23’s picture

Priority: Normal » Critical
voleger’s picture

StatusFileSize
new13.27 KB

Here is the first suggested option of the resolution

voleger’s picture

+++ b/core/lib/Drupal/Core/DrupalKernel.php
@@ -776,7 +776,7 @@ protected function moduleData($module) {
-      $settings = $this->getConfigStorage()->read('simpletest.settings');
...
       $parent_profile = !empty($settings['parent_profile']) ? $settings['parent_profile'] : NULL;

+++ b/core/lib/Drupal/Core/Extension/ExtensionDiscovery.php
@@ -233,7 +233,7 @@ public function setProfileDirectoriesFromSettings() {
-      $testing_profile = \Drupal::config('simpletest.settings')->get('parent_profile');
...
       if ($testing_profile && $testing_profile != $profile) {
         $this->profileDirectories[] = drupal_get_path('profile', $testing_profile);
       }

+++ b/core/lib/Drupal/Core/Extension/ModuleExtensionList.php
@@ -111,7 +111,7 @@ protected function getProfileDirectories(ExtensionDiscovery $discovery) {
-    $parent_profile = $this->configFactory->get('simpletest.settings')->get('parent_profile');
...
     if ($parent_profile && !isset($profiles[$parent_profile])) {
       // In case both profile directories contain the same extension, the

Can we decouple here using some of the mechanisms like hooks or events?

voleger’s picture

  1. +++ b/core/lib/Drupal/Core/Test/FunctionalTestSetupTrait.php
    @@ -107,7 +107,7 @@ protected function prepareSettings() {
         // Add the parent profile's search path to the child site's search paths.
         // @see \Drupal\Core\Extension\ExtensionDiscovery::getProfileDirectories()
    -    $settings['conf']['simpletest.settings']['parent_profile'] = (object) [
    ...
           'value' => $this->originalProfile,
           'required' => TRUE,
         ];
    

    Not really sure about this part. What exactly we can do here?

  2. +++ b/core/tests/Drupal/KernelTests/Config/DefaultConfigTest.php
    @@ -35,14 +35,14 @@ class DefaultConfigTest extends KernelTestBase {
    -   * @todo Figure out why simpletest.settings is not installed.
    ...
       public static $skippedConfig = [
         'locale.settings' => ['path: '],
         'syslog.settings' => ['facility: '],
    

    I guess we are able to fix that todo in that issue.

Other changes are related to the code within simpletest module. So I like the idea (second option from the proposed resolution) to introduce another mechanism to pass those configurations.

mile23’s picture

Status: Active » Needs review
voleger’s picture

Also, I find usages of the simpletest.settings config in the generated settings.php files after running tests locally. So there introduced only the 'parent_profile' key with NULL value. I suspect that is set for specific needs and for all running tests and can be moved in some global testing configuration, but I'm not sure is it works as I think.

mile23’s picture

StatusFileSize
new5.25 KB
+++ b/core/modules/simpletest/src/TestBase.php
@@ -902,7 +902,7 @@ public function run(array $methods = []) {
-    $simpletest_config = $this->config('simpletest.settings');
+    $simpletest_config = $this->config('system.testing_settings');

+++ b/core/modules/system/config/schema/system.schema.yml
@@ -313,6 +313,31 @@ system.theme.global:
+system.testing_settings:
+  type: config_object
+  label: 'Testing'
+  mapping:

I don't think TestBase (and by extension WebTestBase) need to have any of their config moved to the system module. That config should move wherever the simpletest module goes after we deprecate it.

I think what's going on here is that FunctionalTestSetupTrait is sneaking some config into the kernel and the extension handlers so for special cases involving profiles. Functional tests can specify a profile other than the one of the installed site (if there is one).

This should probably be global config for the fixture site in some way, and not belong to any module. In theory, you could run a functional test that doesn't install the system module.

But what we can try here is to just do the same sneaky trick with one of the system config objects. This patch uses system.site and adds test_parent_profile.

We should probably follow-up with getting rid of this system of passing config around for functional tests. We already have #3038513: Move drupal_generate_test_ua() into the test system where we’re trying to deprecate drupal_valid_test_ua(), and maybe we can encapsulate the profile config into that.

Also there’s the simpletest.config_schema_checker service added by FunctionalTestSetupTrait, and also KernelTestBase::containerBuild(). This service registers itself as an event handler, and it (optionally) checks all config changes for validity. It’s only named after simpletest, and doesn’t have any real connection to the module.

Here’s a patch that does these two things:

  1. Just rename the config to [‘system.site’][‘test_parent_profile’].
  2. Just renames the schema checker service to testing.config_schema_checker.

No interdiff for the different approach.

Success on small local test runs... Let's see what happens with the testbot.

mile23’s picture

Applying #10 to a patch which also deletes most of the simpletest module in #3075490-26: Move simpletest module to contrib

mile23’s picture

mile23’s picture

mile23’s picture

Re: #10 and #11: #3075490-26: Move simpletest module to contrib looks very successful, with only fails relating to updates from fixtures which expect to have the simpletest.settings config available. We'll need to change those fixtures in some way eventually.

larowlan’s picture

Issue tags: +Needs change record
+++ b/core/lib/Drupal/Core/Extension/ModuleExtensionList.php
@@ -111,7 +111,7 @@ protected function getProfileDirectories(ExtensionDiscovery $discovery) {
-    $parent_profile = $this->configFactory->get('simpletest.settings')->get('parent_profile');
+    $parent_profile = $this->configFactory->get('system.site')->get('test_parent_profile');

We have no schema for this - does it even need to be config? i.e. could it just be Settings instead? Other than that, this is just moving stuff around - looks good - patch is much smaller than I was expecting.

If we leave it as is (Config) or move it to settings, we need a change-record because there might be someone relying on the previous key.

alexpott’s picture

The parent profile stuff was added in #911354: Tests in profiles/[name]/modules cannot be run and cannot use a different profile for running tests - I think we messed up when we moved it to config. It's not really something that should be configured. Let's move it to settings.php as something in $settings['testing'['parent_profile'] - it is not something that should ever be set by a human being and therefore should not be in config.

If someone is relying on the previous key then I think a CR should suffice - it was never meant to be a configurable thing and even if you do currently set it it will be overridden because we write an override to settings.php so maybe mentioning on the CR is not necessary as it is just noise that complicates for no gain.

alexpott’s picture

Since we do

    $settings['settings']['test_parent_site'] = (object) [
      'value' => $this->originalSite,
      'required' => TRUE,
    ];

maybe $settings['settings']['test_parent_profile'] is better than $settings['testing']['parent_profile']

mile23’s picture

StatusFileSize
new6.04 KB
new4.15 KB

This may come as a shock. :-)

I'm pretty sure we don't need any of this config because of #1808220: Remove run-tests.sh dependency on existing/installed parent site and we can just set the profile for the fixture site without juggling it around during extension discovery.

As a caveat: I keep re-reading this paragraph trying to comprehend whether it's meaningful or not:

+++ b/core/lib/Drupal/Core/Extension/ExtensionDiscovery.php
@@ -229,15 +229,6 @@ public function scan($type, $include_tests = NULL) {
-    // For SimpleTest to be able to test modules packaged together with a
-    // distribution we need to include the profile of the parent site (in
-    // which test runs are triggered).
alexpott’s picture

lolz - smart - yet another reason why profiles should be in configuration :). But I'm not convinced that that is the way to go here. At the moment tests that need this kind of have two profiles (at least for searching for modules) and this would mean that now they only have one. Which means that we've changed the underlying behaviour. On the other hand, this means that the test is way more like a real installation as that can only have a single profile.

alexpott’s picture

Also this will break the profile set by \Drupal\Tests\BrowserTestBase::$profile - which is used in \Drupal\Core\Test\FunctionalTestSetupTrait::installParameters()

mile23’s picture

Also this will break the profile set by \Drupal\Tests\BrowserTestBase::$profile

I tried out a few tests that set $profile and they used the one set. Also the testbot seems happy so far.

Also it's a little weird that TestSetupTrait::$originalProfile is never set anywhere. I guess it really does refer to an installation profile that's currently being tested but is not the fixture.

mile23’s picture

Chatting with @alexpott in slack reveals that we should probably just make it a setting and not config, and also file a follow-up.

We're concerning ourselves with this config: ['conf']['simpletest.settings']['parent_profile']. The value of that config is set to TestSetupTrait::$originalProfile, which is there so that when you're running WebTestBase under the simletest UI, you don't hose your installation profile config.

We're theoretically getting rid of the simpletest module, so we won't have any need for this feature. And that leads us to this followup: #3081501: Remove TestSetupTrait::$originalProfile and test infrastructure which uses it

catch’s picture

#22 sounds like a good plan, no reason for this to be config.

mile23’s picture

StatusFileSize
new5.31 KB
new4.21 KB

Here it is as a setting, and not config. This change is relatively non-disruptive, and the disruption will happen in the followup #3081501: Remove TestSetupTrait::$originalProfile and test infrastructure which uses it

alexpott’s picture

Issue tags: -Needs change record

As per #16 this doesn't need a change record because even if you set this when you run a test it'll be overridden.

larowlan’s picture

+++ b/core/modules/simpletest/src/KernelTestBase.php
@@ -351,7 +351,7 @@ public function containerBuild(ContainerBuilder $container) {
+        ->register('testing.config_schema_checker', ConfigSchemaChecker::class)

+++ b/core/tests/Drupal/KernelTests/KernelTestBase.php
@@ -532,7 +532,7 @@ public function register(ContainerBuilder $container) {
+        ->register('testing.config_schema_checker', ConfigSchemaChecker::class)

should we add these to simpletest.services.yml or a SimpletestServiceProvider as aliases with a deprecated tag?

At the very least we probably need a change record for them as someone might be using these service IDs in their test..no idea why but it could happen.

alexpott’s picture

Re #26 well these are event listeners which I think are explicitly not API

mile23’s picture

Issue tags: -Needs followup

Following the principle that too much documentation is better than not enough, added CR: https://www.drupal.org/node/3082134

NF tag from #13 referred to trying to encapsulate this config in drupal_valid_test_ua(): #3038513: Move drupal_generate_test_ua() into the test system But the plan looks like we're just going to remove it: #3081501: Remove TestSetupTrait::$originalProfile and test infrastructure which uses it

larowlan’s picture

Status: Needs review » Reviewed & tested by the community

In that case

  • catch committed 5db19a0 on 8.8.x
    Issue #3080482 by Mile23, voleger, alexpott, larowlan: Decouple...
catch’s picture

Status: Reviewed & tested by the community » Fixed

Committed 5db19a0 and pushed to 8.8.x. Thanks!

Will unpostpone the follow-up.

wim leers’s picture

Published the change record: https://www.drupal.org/node/3082134

xjm’s picture

Priority: Critical » Major

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.