Problem/Motivation

In #3350972: [random test failure] Drupal\Tests\layout_builder\FunctionalJavascript\LayoutBuilderUiTest::testReloadWithNoSections() @nod_ introduced a "fix" for problems with the correct opening of an off-canvas dialog which potentially could fix all JS random test failures for tests using the drupal.dialog.off_canvas somehow.

The basic test is:

  1. Unskip BlockFormMessagesTest::testValidationMessage().
  2. Run _only_ now unskipped BlockFormMessagesTest::testValidationMessage() a lot of times (usually we go for 1500x) as-is.
  3. Run _only_ now unskipped BlockFormMessagesTest::testValidationMessage() a lot of times (usually we go for 1500x) _without_ the changes in #3350972: [random test failure] Drupal\Tests\layout_builder\FunctionalJavascript\LayoutBuilderUiTest::testReloadWithNoSections().
  4. If 2., passes and 3. doesn't, we can safely turn the test back on again.

Per @xjm in #3353085-4: [meta] Determine impact of [#3350972] fix in off-canvas.js on currently disabled FunctionalJavascript tests we now need to go for a 5000-8000 times run for both patches.

Steps to reproduce

Proposed resolution

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

CommentFileSizeAuthor
#3 3353088-3.patch958 bytesspokje
#2 BlockFormMessagesTest_1500x.patch2.71 KBspokje
#2 BlockFormMessagesTest-no_#3350972-1500x-should_fail.patch3.2 KBspokje

Comments

Spokje created an issue. See original summary.

spokje’s picture

spokje
Primary language Dutch
commented
StatusFileSize
new BlockFormMessagesTest-no_#3350972-1500x-should_fail.patch3.2 KB
new BlockFormMessagesTest_1500x.patch2.71 KB
spokje’s picture

spokje
Primary language Dutch
commented
Status: Active » Needs review
StatusFileSize
new 3353088-3.patch958 bytes
spokje’s picture

spokje
Primary language Dutch
commented
Title: #3268244: [random test failure] Un-skip and fix BlockFormMessagesTest::testValidationMessage() » [random test failure] Un-skip and fix BlockFormMessagesTest::testValidationMessage()

The last submitted patch, 2: BlockFormMessagesTest-no_#3350972-1500x-should_fail.patch, failed testing. View results

spokje’s picture

spokje
Primary language Dutch
commented
Title: [random test failure] Un-skip and fix BlockFormMessagesTest::testValidationMessage() » [random test failure] Try to un-skip and fix BlockFormMessagesTest::testValidationMessage() in context of [#3353085]
smustgrave’s picture

smustgrave commented
Status: Needs review » Reviewed & tested by the community
Issue tags: +Needs Review Queue Initiative

The condition mentioned in the issue summary seems to have been met. 1500X with the fix passes, 1500x without the fix fails.

spokje’s picture

spokje
Primary language Dutch
commented
Issue summary: View changes

The last submitted patch, 2: BlockFormMessagesTest-no_#3350972-1500x-should_fail.patch, failed testing. View results

The last submitted patch, 2: BlockFormMessagesTest-no_#3350972-1500x-should_fail.patch, failed testing. View results

The last submitted patch, 2: BlockFormMessagesTest-no_#3350972-1500x-should_fail.patch, failed testing. View results

spokje’s picture

spokje
Primary language Dutch
commented
Assigned: Unassigned » spokje
Status: Reviewed & tested by the community » Needs work
spokje’s picture

spokje
Primary language Dutch
commented
Assigned: spokje » Unassigned
Status: Needs work » Reviewed & tested by the community

Both should_fail and non_fail patch run 5 times. That's 7500 individual tests per patch.

Back to RTBC.

xjm’s picture

xjm
she/her
Primary language English
commented

I only see the one 1500x job for each? Adding a few more.

spokje’s picture

spokje
Primary language Dutch
commented

One job, ran 5 times per patch: https://www.drupal.org/pift-ci-job/2639217 and https://www.drupal.org/pift-ci-job/2639224 (scroll all the way down for the latter).

xjm’s picture

xjm
she/her
Primary language English
commented

Fixing attribution.

  • catch committed b4d06001 on 10.1.x 
    Issue #3353088 by Spokje: [random test failure] Try to un-skip and fix...
catch’s picture

catch
he/him
Primary language English
commented
Status: Reviewed & tested by the community » Patch (to be ported)

Committed/pushed to 10.1.x, thanks!
Moving to 10.0.x for backport later.

  • catch committed d5178570 on 10.0.x 
    Issue #3353088 by Spokje: [random test failure] Try to un-skip and fix...

  • catch committed 4ac40b00 on 9.5.x 
    Issue #3353088 by Spokje: [random test failure] Try to un-skip and fix...
catch’s picture

catch
he/him
Primary language English
commented
Version: 10.0.x-dev » 9.5.x-dev
Status: Patch (to be ported) » Fixed

Cherry-picked to 10.0.x and 9.5.x, thanks!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.