Come together with the global Drupal community in Rotterdam, 28 Sept – 1 Oct 2026. Sessions, contribution, connection, and Early Bird savings until 8 June.Problem/Motivation
In #3350972: [random test failure] Drupal\Tests\layout_builder\FunctionalJavascript\LayoutBuilderUiTest::testReloadWithNoSections() @nod_ introduced a "fix" for problems with the correct opening of an off-canvas dialog which potentially could fix all JS random test failures for tests using the library drupal.dialog.off_canvas somehow.
This issue is a META to track-n-test all currently skipped FunctionalJavascript tests that make use of that library.
The basic test for each issue would be:
- Unskip test.
- Run _only_ now unskipped test a lot of times (
usually we go for 1500x) as-is. - Run _only_ now unskipped test a lot of times (
usually we go for 1500x) _without_ the changes in #3350972: [random test failure] Drupal\Tests\layout_builder\FunctionalJavascript\LayoutBuilderUiTest::testReloadWithNoSections(). - If 2, passes and 3 doesn't, we can safely turn the test back on again.
Per @xjm in #3353085-4: [meta] Determine impact of [#3350972] fix in off-canvas.js on currently disabled FunctionalJavascript tests we now need to go for a 5000-8000 times run for both patches.
Currently the library drupal.dialog.off_canvas is used in:
- core/modules/layout_builder/layout_builder.libraries.yml, so we need to check all skipped tests in core/modules/layout_builder/tests/src/FunctionalJavascript.
- core/themes/stable9/stable9.info.yml, so we need to check all skipped FunctionalJavascript tests that use that theme.
- Just went through every skipped FunctionalJavascript test.
Comments
Comment #2
spokjeComment #3
spokjeComment #4
xjmActually, 1500 runs is not statistically sufficient to prove something is fixed. Each test should be queued against a MySQL or Maria database 5000-8000 times. This can be accomplished by queuing a patch that runs the test 1000x for the required number of times. Don't use Postgres or SQLite databases for this.
Comment #5
spokjeOk, I was under the impression we use 1500x as the magic number, I'll just requeue the 1500x patches 4 more times.
However, that leaves us with patches that are too slow for a 1500x run in the Drupal CI environment and hit the timeout like #3353103: [no random test failure] Try to un-skip and fix LayoutBuilderDisableInteractionsTest::assertContextualLinksClickable in context of [#3353085] where I had to down-scale to a 500x run.
Do we need to requeue those patches (at least) 9 more times?
Comment #6
spokjeComment #7
spokjeComment #8
spokjeComment #9
spokjeIf so, can I tag those issues as "Novice" and the person that requeues it for the required number of reruns will receive credit for that?
Comment #10
xjmYep I have queued 10 or 15 jobs in the past for tests that are too slow to run more per job.
I can just queue the jobs now; not sure why that would be a novice task.
Comment #11
xjmFixing attribution.
Comment #12
spokjeComment #13
catchWe might have resolved #3210432: [random test failure] LayoutBuilderQuickEditTest::testQuickEditIgnoresDuplicateFields() too (9.5 only).
Comment #14
spokjeSadly not: https://www.drupal.org/project/drupal/issues/3210432#comment-15007163
Comment #15
spokjeClosing as all of its children are in an end-status.
Comment #16
quietone commented@Spokje, thank you for managing this and working on the child issues !