Problem/Motivation

This is a follow-on to #2745123: Simpletest module crashes on cleanup, uninstall which will, we hope, find non-disruptive solutions to the problem of class loading classes under core/tests during runtime. This includes the problem of simpletest_clean_environment() needing to access TestBase.

In this issue we can refactor the simpletest cleanup code to be more maintainable and clear, with inversion of control, in a somewhat more disruptive way for 8.3.x.

simpletest_cleanup_environment() is used by both the Simpletest UI and run-tests.sh. This makes it a good candidate for refactoring to improve the testing system moving forward.

Proposed resolution

Refactor the following functions into a helper class called something like EnvironmentCleaner:

  • simpletest_clean_environment()
  • simpletest_clean_database()
  • simpletest_clean_temporary_directories()
  • simpletest_clean_results_table()

Mark these functions as deprecated.

Remaining tasks

Turn the EnvironmentCleaner class into a proper service. This would involve turning the TestDatabase class into a service or parameter service as well which might be outside the scope of this issue.

User interface changes

API changes

Data model changes

Comments

Mile23 created an issue. See original summary.

mile23’s picture

Title: Turn simpletest_clean*() functions into a helper class » Turn simpletest_clean*() functions into a helper class/service
Status: Active » Needs review
StatusFileSize
new10.36 KB

First pass at this effort. Discuss. :-)

mile23’s picture

joelpittet’s picture

Status: Needs review » Needs work
Issue tags: +Needs re-roll

It looks to not apply anymore

drugan’s picture

StatusFileSize
new10.71 KB
new1.1 KB

Re-rolled #2.

I think that this solution for crashes when cleaning environment is good alternative to another one.

The only deficiency is that it is a bit heavier than the one on the link above.

drugan’s picture

Status: Needs work » Needs review
drugan’s picture

Issue tags: -Needs re-roll

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

mile23’s picture

StatusFileSize
new10.19 KB
new897 bytes

Rerolled, amended to reflect changes in #2745123: Simpletest module crashes on cleanup, uninstall

mile23’s picture

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

dawehner’s picture

  1. +++ b/core/modules/simpletest/simpletest.module
    @@ -9,6 +9,7 @@
    +use Drupal\simpletest\EnvironmentCleaner;
    
    @@ -644,68 +645,24 @@ function simpletest_generate_file($filename, $width, $lines, $type = 'binary-tex
     function simpletest_clean_environment() {
    ...
    +  $cleaner = new EnvironmentCleaner(\Drupal::root(), TestDatabase::getConnection(), \Drupal::database(), \Drupal::translation());
    +  $cleaner->cleanEnvironment();
     }
    

    It would be nice to not have this below the simpletest namespace but under the generic test bit.

  2. +++ b/core/modules/simpletest/simpletest.module
    @@ -644,68 +645,24 @@ function simpletest_generate_file($filename, $width, $lines, $type = 'binary-tex
     function simpletest_clean_database() {
    ...
     function simpletest_clean_temporary_directories() {
    
    @@ -718,29 +675,8 @@ function simpletest_clean_temporary_directories() {
     function simpletest_clean_results_table($test_id = NULL) {
    

    We should throw deprecation notices and have a @deprecated tags :(

  3. +++ b/core/modules/simpletest/src/EnvironmentCleaner.php
    @@ -0,0 +1,174 @@
    + * @todo: Promote to actual service status. This will require promoting
    + *   TestDatabase to a service as well.
    

    I don't think this is something we have to do. We can do dependency injection without it being a service, aka. be somehow part of a module. What about dropping this todo?

  4. +++ b/core/modules/simpletest/src/EnvironmentCleaner.php
    @@ -0,0 +1,174 @@
    + * @todo: Inject config and cache services.
    

    Honestly, test code has different requirements than production code IMHO. One could make up a scope which is minimal aka. just moving code, but sure, let's create a followup for that?

mile23’s picture

Title: Turn simpletest_clean*() functions into a helper class/service » Turn simpletest_clean*() functions into a helper class
Issue tags: +Needs change record, +@deprecated
StatusFileSize
new16.9 KB
new9.1 KB

#13.1: Done.

#13.2: Trigger errors should be a follow-up, when we remove usages. Otherwise all tests will always fail. :-) @deprecation notices added.

#13.3: Agreed. If it was coupled to simpletest module, then maybe it should be a service, but since it's not, it shouldn't be. But it takes a lot of other services as arguments. See next point, too.

#13.4:

Honestly, test code has different requirements than production code IMHO. One could make up a scope which is minimal aka. just moving code, but sure, let's create a followup for that?

Test code has stricter requirements. :-)

Seriously, we should be making a class that performs all these functions without, for instance, asking \Drupal::config() if it should. Then callers should be responsible for asking config and reporting results. In run-tests.sh we call simpletest_clean_results_table(), which shouldn't require config, right?

But that's not an easy refactor and the main goal here is to get it out of simpletest. Then someone can isolate it from dependencies if it becomes important.

So with that in mind, I injected the messenger service instead of calling drupal_set_message(). :-)

We still use file_unmanaged_delete_recursive(), which requires an .inc file, but I found this: #2244513: Move the unmanaged file APIs to the file_system service (file.inc)

mile23’s picture

Issue tags: +Needs followup

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

mile23’s picture

Some planning happening in this meta.

mile23’s picture

StatusFileSize
new24.48 KB

Needed a reroll, but I ended up rearchitecting a little bit.

Adds a service to the simpletest module called environment_cleaner. Within the context of the simpletest module, this service behaves with BC. It's a service so it can do things like display messages through the messenger service.

The service inherits from the real helper class, which is Drupal\Core\Test\EnvironmentCleaner. This can run in many different contexts. Its only complicated dependency is file_service.

This patch does a poor job of deprecating all the simpletest_clean*() functions. It only demonstrates the changes and will find deprecated usages in core, but there shouldn't be any.

If there is some consensus on this, we can work on the deprecation messages and so forth.

mile23’s picture

StatusFileSize
new24.37 KB
new171 bytes

Forgot one important change.

andypost’s picture

Related issues: +#2838474: Remove dependency of "file_system" service on "logger"

Btw cmi2 also "figured out" environment for config
The file_service also depends on config and stream wrappers which is blocked on related

mile23’s picture

The only reason we need file_system here is to perform a recursive delete. We don't need any logging or really any other service dependencies. We know where the directory is, and it should never be a stream wrapper type URL.

Ideally, we'd use code that has nothing to do with core, because we're testing core, which might break file_system.

In any other project, symfony/filesystem to the rescue. :-)

lendude’s picture

StatusFileSize
new24.39 KB

Needed a reroll.

No changes, just a reroll for now.

mile23’s picture

Issue tags: -Needs change record

Added CR: https://www.drupal.org/node/3076634

Updated deprecation messages.

Added a deprecation test, and also some unit testing for Drupal\Core\Test\EnvironmentCleaner.

As always: It's a little tricky to test code that changes test results.

mile23’s picture

StatusFileSize
new31.24 KB
new10.81 KB

And perhaps I should include a patch.

mile23’s picture

StatusFileSize
new31.39 KB
new4.3 KB

CS.

lendude’s picture

StatusFileSize
new31.33 KB

Needed another reroll, also got rid of an unused use while doing it.

lendude’s picture

Really just some nits, the rest looks really nice.

  1. +++ b/core/lib/Drupal/Core/Test/EnvironmentCleaner.php
    @@ -0,0 +1,189 @@
    + * Clean up after tests.
    

    This could be a little more descriptive, 'Helper class for cleaning generated test environments.'. I don't know, something like that?

  2. +++ b/core/lib/Drupal/Core/Test/EnvironmentCleaner.php
    @@ -0,0 +1,189 @@
    +   * This could be the same as $testDatabase, or it could be different.
    

    Either leave this out, or explain how/when it's the same or when it's different

  3. +++ b/core/lib/Drupal/Core/Test/EnvironmentCleaner.php
    @@ -0,0 +1,189 @@
    +  /**
    +   *
    +   * @var \Symfony\Component\Console\Output\OutputInterface
    +   */
    +  protected $output;
    

    Needs a description

  4. +++ b/core/lib/Drupal/Core/Test/EnvironmentCleaner.php
    @@ -0,0 +1,189 @@
    +  /**
    +   *
    +   * @return int
    +   *   The number of tables that were removed.
    +   */
    +  protected function doCleanDatabase() {
    

    Needs a description

  5. +++ b/core/lib/Drupal/Core/Test/EnvironmentCleaner.php
    @@ -0,0 +1,189 @@
    +  /**
    +   *
    +   * @return int
    +   *   The count of temporary directories removed.
    +   */
    +  protected function doCleanTemporaryDirectories() {
    

    Needs a description

  6. +++ b/core/modules/simpletest/simpletest.services.yml
    @@ -1,9 +1,17 @@
    -    # @todo Change this service so that it uses \Drupal\Core\Test\TestDiscovery
    -    # in https://www.drupal.org/node/1667822
    +    # Drupal\simpletest\TestDiscovery is deprecated, but we still want to use it
    +    # as the service here to support BC hooks.
    

    Unrelated change?

  7. +++ b/core/modules/simpletest/simpletest.services.yml
    @@ -1,9 +1,17 @@
    +
    +  environment_cleaner_factory:
    +    class: Drupal\simpletest\EnvironmentCleanerFactory
    +    arguments: ['@service_container']
    +  environment_cleaner:
    +    class: Drupal\simpletest\EnvironmentCleanerService
    +    factory: 'environment_cleaner_factory:createCleaner'
    +
    

    Extra newlines above and below are not needed.

Status: Needs review » Needs work

The last submitted patch, 27: 2800267-27.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

mile23’s picture

Status: Needs work » Needs review
StatusFileSize
new31.5 KB
new4.1 KB

Thanks.

Addressed #28, fixed test fail.

lendude’s picture

Status: Needs review » Needs work
StatusFileSize
new36.02 KB

@Mile23++

Tested this some more. The output on the bot seems fine, but when running 'clean environment' via the Simpletest UI I get the following feedback:

That seems to work better without the patch.

Also I needed a cache rebuild for the clean up to work in the Simpletest UI because of the new service being introduced. So I guess we need a post_update to force a cache rebuild?

mile23’s picture

Status: Needs work » Needs review
StatusFileSize
new31.43 KB
new2.6 KB

Thanks, @Lendude.

Updated with proper formatPlural() format.

lendude’s picture

Status: Needs review » Reviewed & tested by the community
StatusFileSize
new28.51 KB


Looks good now.

Feedback has been addressed. So one question remains: do we need an update function to force a cache clear? Seems excessive, but lets see if others agree.

mondrake’s picture

+++ b/core/lib/Drupal/Core/Test/EnvironmentCleanerInterface.php
@@ -0,0 +1,43 @@
+<?php
+
+namespace Drupal\Core\Test;
+
+interface EnvironmentCleanerInterface {
+
+  /**
+   * Removes all test-related database tables and directories.
+   *
+   * This method removes fixture files and database entries from the system
+   * under test.
+   *
+   * @param bool $clear_results
+   *   (optional) Whether to clear the test results database. Defaults to TRUE.
+   * @param bool $clear_temp_directories
+   *   (optional) Whether to clear the test site directories. Defaults to TRUE.
+   * @param bool $clear_database
+   *   (optional) Whether to clean up the fixture database. Defaults to TRUE.
+   */
+  public function cleanEnvironment($clear_results = TRUE, $clear_temp_directories = TRUE, $clear_database = TRUE);
+
+  /**
+   * Remove database entries left over in the fixture database.
+   */
+  public function cleanDatabase();
+
+  /**
+   * Finds all leftover fixture site directories and removes them.
+   */
+  public function cleanTemporaryDirectories();
+
+  /**
+   * Clears test result tables from the results database.
+   *
+   * @param $test_id
+   *   Test ID to remove results for, or NULL to remove all results.
+   *
+   * @return int
+   *   The number of results that were removed.
+   */
+  public function cleanResultsTable($test_id = NULL);
+
+}

Would it make sense to mark these methods @internal since their signatures might change later while working with #3075608: Introduce TestRun objects and refactor TestDatabase to be testable? I think these is not to be considered public API.

mile23’s picture

mile23’s picture

StatusFileSize
new32.1 KB
new464 bytes

Updated for CS. Leaving at RTBC because it's a very minor change.

mile23’s picture

Issue summary: View changes

Updated IS to be more in line with current goals.

mile23’s picture

Issue tags: -Needs followup

Not entirely sure why I added 'needs followup' in #15, presumably because of #2244513: Move the unmanaged file APIs to the file_system service (file.inc) That issue has been fixed and closed, so I don't think we need that tag any more.

mile23’s picture

Priority: Normal » Critical
larowlan’s picture

Priority: Critical » Normal
Issue summary: View changes
Issue tags: +Needs followup
larowlan’s picture

Status: Reviewed & tested by the community » Fixed

Committed 914d47f and pushed to 8.8.x. Thanks!

  • larowlan committed 914d47f on 8.8.x
    Issue #2800267 by Mile23, Lendude, drugan, dawehner, mondrake: Turn...
mile23’s picture

Yay! 🎉

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.