Problem/Motivation

InstallUninstallTest seems to take longer than it should.

Proposed resolution

Find and eliminate the reasons for the delay.

InstallUninstallTest in seconds:

Mode HEAD HEAD + 2659940-141 WTB
Only InstallUninstallTest 827 835 (HEAD +8) 622 (HEAD -205)
With all tests 1199 1943 (HEAD +744) 981 (HEAD -218)
With all tests + TTL 1 1208 1222 (HEAD + 14)

See also #2659940-162: Extension System, Part III: ThemeExtensionList and ThemeEngineExtensionList.

Remaining tasks

User interface changes

none

API changes

none

Data model changes

none

CommentFileSizeAuthor
#12 2765271-6__ttl-1__with-time.patch2.87 KBAnonymous (not verified)
#11 2659940-141__2765271-6__ttl-1__with-time.patch81.76 KBAnonymous (not verified)
#10 2659940-141__2765271-6__ttl-10__with-time.patch81.76 KBAnonymous (not verified)
#9 wtb-2659940-141-all-tests-with-time.patch80.98 KBAnonymous (not verified)
#8 btb-2659940-141-all-tests-with-time.patch80.27 KBAnonymous (not verified)
#8 wtb-InstallUninstallTest-all-tests-with-time.patch2.09 KBAnonymous (not verified)
#5 script-metrics-test-time.php_.txt913 bytesAnonymous (not verified)
#5 wtb-InstallUninstallTest-all-tests-with-time.patch2.09 KBAnonymous (not verified)
#5 btb-2659940-141-all-tests-with-time.patch80.27 KBAnonymous (not verified)
#3 all-tests-with-time.patch1.38 KBAnonymous (not verified)
#2 btb-2659940-141-InstallUninstallTest.patch79.4 KBAnonymous (not verified)
#2 wtb-InstallUninstallTest.patch1.22 KBAnonymous (not verified)
#2 btb-head-InstallUninstallTest.patch529 bytesAnonymous (not verified)

Comments

Anonymous’s picture

Anonymous (not verified) commented

vaplas created an issue. See original summary.

Anonymous’s picture

Anonymous (not verified) commented
Status: Active » Needs review
Parent issue: #2971472: SettingsTrayBlockFormTest is taking 5 minutes to run »
Related issues: +#2930072: Module: Convert system functional tests to phpunit, +#2659940: Extension System, Part III: ThemeExtensionList and ThemeEngineExtensionList
StatusFileSize
new btb-head-InstallUninstallTest.patch529 bytes
new wtb-InstallUninstallTest.patch1.22 KB
new btb-2659940-141-InstallUninstallTest.patch79.4 KB

Edit:
Check only one InstallUninstallTest :

  • WTB: 10 min 22 sec (622 sec)
  • BTB HEAD: 13 min 47 sec (827 sec)
  • BTB HEAD + 2659940-141: 13 min 54 sec (834 sec)
Anonymous’s picture

Anonymous (not verified) commented
StatusFileSize
new all-tests-with-time.patch1.38 KB

Edit:
Check InstallUninstallTest when run all tests:

  • BTB HEAD: 19 min 59 sec (1199 sec)
Mixologic’s picture

Mixologic
He/Him/His
Primary language English
Location Portland, OR
commented

I see you have found the dragon in the testing cave. This test is a monster.

Anonymous’s picture

Anonymous (not verified) commented
Related issues: +#2421479: Use a bootstrap.php file to load the most used classes in one go, +#1818628: Use Composer's optimized ClassLoader for Core/Component classes
StatusFileSize
new btb-2659940-141-all-tests-with-time.patch80.27 KB
new wtb-InstallUninstallTest-all-tests-with-time.patch2.09 KB
new script-metrics-test-time.php_.txt913 bytes

Yep, top-5 dragons in #3 by seconds:

  1. [Drupal\Tests\system\Functional\Module\InstallUninstallTest] => 1199
  2. [Drupal\Tests\rest\Functional\Views\StyleSerializerTest] => 539
  3. [Drupal\Tests\config_translation\Functional\ConfigTranslationUiTest] => 502
  4. [Drupal\Tests\system\Functional\Update\UpdatePathTestBaseFilledTest] => 430
  5. [Drupal\Tests\media\Functional\MediaUiFunctionalTest] => 334

However in #2 InstallUninstallTest is only 827 seconds. So the additional 372 seconds when running all the tests.

The #2659940-141: Extension System, Part III: ThemeExtensionList and ThemeEngineExtensionList slows down the running all tests even more. For example, InstallUninstallTest will be near 1942 seconds! But in #2 it is only 835 seconds (almost like HEAD).

Difference between #138 and #141 is:

+++ b/core/lib/Drupal/Core/Extension/ThemeExtensionList.php
@@ -140,11 +140,6 @@ protected function doList() {
-    $this->registerThemeNamespaces($themes);

@@ -183,25 +178,6 @@ protected function fillInSubThemeData(array &$themes, array $sub_themes) {
-    foreach ($themes as $theme) {
-      // Only installed themes should be registered.
-      if ($theme->status) {
-        drupal_classloader_register($theme->getName(), $theme->getPath());
-      }

But it leads to a colossal subsidence of some tests when full tests run (not only for InstallUninstallTest, it is just good example).

Perhaps this is due to APCu fragmentation, or opcache, or opcode, or some other leak of some clever term :)

The last submitted patch, 5: btb-2659940-141-all-tests-with-time.patch, failed testing. View results

Status: Needs review » Needs work

The last submitted patch, 5: wtb-InstallUninstallTest-all-tests-with-time.patch, failed testing. View results

Anonymous’s picture

Anonymous (not verified) commented
StatusFileSize
new wtb-InstallUninstallTest-all-tests-with-time.patch2.09 KB
new btb-2659940-141-all-tests-with-time.patch80.27 KB

Edit:
Check InstallUninstallTest when run all tests:

  • WTB: 16 min 21 sec (981 sec)
  • BTB HEAD + 2659940-141: 32 min 23 sec (1943 sec)
Anonymous’s picture

Anonymous (not verified) commented
StatusFileSize
new wtb-2659940-141-all-tests-with-time.patch80.98 KB

Edit:
Check InstallUninstallTest when run all tests:

  • WTB + 2659940-141: 16 min 33 sec (992 sec)
Anonymous’s picture

Anonymous (not verified) commented
StatusFileSize
new 2659940-141__2765271-6__ttl-10__with-time.patch81.76 KB

Edit:
Typo in patch name. Not ttl 10, but ttl 30 here.

Check InstallUninstallTest when run all tests:

  • BTB HEAD + 2659940-141 + 2765271-6 + ttl-30: 29 min 7 sec (1747 sec)
Anonymous’s picture

Anonymous (not verified) commented
StatusFileSize
new 2659940-141__2765271-6__ttl-1__with-time.patch81.76 KB

Edit:

Check InstallUninstallTest when run all tests:

  • BTB HEAD + 2659940-141 + 2765271-6 + TTL 1: 20 min 22 sec (1222 sec)
Anonymous’s picture

Anonymous (not verified) commented
StatusFileSize
new 2765271-6__ttl-1__with-time.patch2.87 KB

Edit:

Check InstallUninstallTest when run all tests:

  • BTB HEAD + 2765271-6 + TTL 1: 20 min 8 sec (1208 sec)

📌 TTL 1 eliminates the difference in execution time between the HEAD and #2659940-141: Extension System, Part III: ThemeExtensionList and ThemeEngineExtensionList when all tests run. Woot!

Anonymous’s picture

Anonymous (not verified) commented
Issue summary: View changes

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

catch’s picture

catch
he/him
Primary language English
commented
Priority: Normal » Major

Rediscovered this issue via #3386091: [PP-1] Reduce test pipeline times to around 18 minutes and always report back unit tests. InstallUninstallTest singlehandedly adds about 20% to test times (once a full test run on gitlab takes 17 minutes, InstallUninstallTest is responsible for 1 or 2 of that because it's just so slow that it's still running when everything else has finished). I think it's time to drop it. We've got a lot more test coverage than when it's added, and the config API means modules do a lot less API calls on install, they can ship config for a lot of things that used to require PHP.

longwave’s picture

longwave
he/him
Primary language English
Location UK
commented
Status: Needs work » Closed (duplicate)

InstallUninstallTest was removed in #3386458: Add GenericModuleTestBase and use it to test general module things