Problem/Motivation

Drupal\Tests\system\FunctionalJavascript\ThemeSettingsFormTest::testFormSettingsSubmissionHandler fails intermittently on MRs that don't touch anything it covers.

Seen on the docs-only MR for #3624887: Remove obsolete hook_extension() and hook_render_template() documentation (it changes only theme.api.php), job https://git.drupalcode.org/issue/drupal-3624887/-/jobs/12520251:

Theme Settings Form (Drupal\Tests\system\FunctionalJavascript\ThemeSettingsForm)
 ✘ Form settings submission handler with test·theme.theme
   ├ Behat\Mink\Exception\ElementNotFoundException: Form hidden field with id|name|value "custom_logo[fids]" not found.
   │ /builds/core/tests/Drupal/Tests/WebAssert.php:681
   │ /builds/core/modules/system/tests/src/FunctionalJavascript/ThemeSettingsFormTest.php:76
 ✔ Form settings submission handler with test·theme-settings.php

The other data set passed in the same run. The same test was also reported as a random failure in #3620912: Deprecate user.module cancel methods, job https://git.drupalcode.org/project/drupal/-/jobs/12026261, at ThemeSettingsFormTest.php:85 in the version of the file at the time.

The likely cause is a race after the form submit

$page->pressButton('Save configuration');
\Drupal::entityTypeManager()->getStorage('file')->resetCache();

// Assert the uploaded file is saved as permanent.
$image_field = $this->assertSession()->hiddenFieldExists('custom_logo[fids]');

In a FunctionalJavascript test, pressButton() doesn't wait for the page that the submit loads. If the assertion runs while the browser is between the old page and the new one, custom_logo[fids] isn't in the DOM and the test fails. The race can also go the other way: the assertion matches the field on the page from before the submit, and assertTrue($file->isPermanent()) can then run before the submit has been processed on the server.

Steps to reproduce

Timing-dependent, so it doesn't fail on every run. Run ThemeSettingsFormTest repeatedly on a loaded CI runner. It fails at line 76 some of the time.

Proposed resolution

Wait for the submit to finish before checking the field, for example by waiting for the confirmation message:

$page->pressButton('Save configuration');
$assert_session->statusMessageContainsAfterWait('The configuration options have been saved.');

For the same reason, also assert the return value of the waitForButton('custom_logo_remove_button') call before the upload assertion. As written, a slow upload would fail one line later with a less clear error.

Remaining tasks

  • Confirm the fix with repeated runs of the test.
  • MR.

User interface changes

N/A

Introduced terminology

N/A

API changes

N/A

Data model changes

N/A

Release notes snippet

N/A

AI disclosure

Claude Code traced this failure while I was reviewing #3624887: Remove obsolete hook_extension() and hook_render_template() documentation and assisted me while drafting this issue.

Issue fork drupal-3626907

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

matthand created an issue. See original summary.

matthand’s picture

Assigned: Unassigned » matthand
Status: Active » Needs review

I made two small edits in the test that were most likely the causes of the random test failure due to race conditions that were possible. I ran a repeat test on the class for 100 cycles and it passed every one. I expect the test flakiness is now resolved. MR is ready for review. TYSM!

matthand’s picture

matthand’s picture

Ironically there's an unrelated random test failure on this issues MR, so I filed a new issue for it #3626913: Random test failure on ManageFieldsTest::testAddField fails with MoveTargetOutOfBounds.