Problem/Motivation

In #3350972: [random test failure] Drupal\Tests\layout_builder\FunctionalJavascript\LayoutBuilderUiTest::testReloadWithNoSections() @nod_ introduced a "fix" for problems with the correct opening of an off-canvas dialog which potentially could fix all JS random test failures for tests using the drupal.dialog.off_canvas somehow.

The basic test is:

  1. Unskip \Drupal\Tests\layout_builder\FunctionalJavascript\InlineBlockPrivateFilesTest::testPrivateFiles.
  2. Run _only_ now unskipped \Drupal\Tests\layout_builder\FunctionalJavascript\InlineBlockPrivateFilesTest::testPrivateFiles a lot of times (usually we go for 1500x) as-is.
  3. Run _only_ now unskipped \Drupal\Tests\layout_builder\FunctionalJavascript\InlineBlockPrivateFilesTest::testPrivateFiles a lot of times (usually we go for 1500x) _without_ the changes in #3350972: [random test failure] Drupal\Tests\layout_builder\FunctionalJavascript\LayoutBuilderUiTest::testReloadWithNoSections().
  4. If 2., passes and 3. doesn't, we can safely turn the test back on again.

Per @xjm in #3353085-4: [meta] Determine impact of [#3350972] fix in off-canvas.js on currently disabled FunctionalJavascript tests we now need to go for a 5000-8000 times run for both patches.

Steps to reproduce

Proposed resolution

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

Comments

Spokje created an issue. See original summary.

spokje’s picture

Status: Active » Needs review
StatusFileSize
new774 bytes
spokje’s picture

catch’s picture

Status: Needs review » Reviewed & tested by the community

That is very encouraging.

spokje’s picture

Indeed, but every silver lining needs a cloud: By the looks of it, we should be able to unskip all/a lot of Layout Builder JS tests, which will add somewhere around 5 minutes to a full test run by my estimates.

spokje’s picture

Issue summary: View changes

spokje’s picture

Assigned: Unassigned » spokje
Status: Reviewed & tested by the community » Needs work
spokje’s picture

Assigned: spokje » Unassigned
Status: Needs work » Reviewed & tested by the community

Both should_fail and non_fail patch run 5 times. That's 7500 individual tests
per patch.

Back to RTBC.

  • catch committed d997fd07 on 10.1.x
    Issue #3353096 by Spokje: [random test failure] Try to un-skip and fix...
catch’s picture

Status: Reviewed & tested by the community » Fixed

Alright one thing at a time!

Committed/pushed to 10.1.x, thanks!

Since we're going to try to commit one of these per day, skipping the backport for now, let's see how everything works together first (if at all, we could also just leave them skipped in earlier branches).

xjm’s picture

Title: [random test failure] Try to un-skip and fix InlineBlockPrivateFilesTest::testPrivateFiles() in context of [#3353085] » [backport to 9.4.x] [random test failure] Try to un-skip and fix InlineBlockPrivateFilesTest::testPrivateFiles() in context of [#3353085]
Version: 10.1.x-dev » 9.4.x-dev
Status: Fixed » Patch (to be ported)

Setting PTPB per discussion with @catch.

  • catch committed 2a216f31 on 10.1.x
    Issue #3353096 by Spokje, catch, xjm: [backport to 9.4.x] [random test...
catch’s picture

Just did a follow-up commit for this:

diff --git a/core/modules/layout_builder/tests/src/FunctionalJavascript/InlineBlockPrivateFilesTest.php b/core/modules/layout_builder/tests/src/FunctionalJavascript/InlineBlockPrivateFilesTest.php
index 6e491cc97b..95f213ac93 100644
--- a/core/modules/layout_builder/tests/src/FunctionalJavascript/InlineBlockPrivateFilesTest.php
+++ b/core/modules/layout_builder/tests/src/FunctionalJavascript/InlineBlockPrivateFilesTest.php
@@ -62,7 +62,6 @@ protected function setUp(): void {
    * Tests access to private files added to inline blocks in the layout builder.
    */
   public function testPrivateFiles() {
-    // Skipped due to frequent random test failures.
     $assert_session = $this->assertSession();
     LayoutBuilderEntityViewDisplay::load('node.bundle_with_section_field.default')
       ->enableLayoutBuilder()

spokje’s picture

Ouch, sloppy work from me there, thanks @catch.

gauravvvv’s picture

Status: Patch (to be ported) » Needs review
StatusFileSize
new743 bytes

I have provided Patch for 9.4.x. please review

smustgrave’s picture

Status: Needs review » Patch (to be ported)

Why are we adding to 9.4.x?

xjm’s picture

Version: 9.4.x-dev » 10.1.x-dev
Status: Patch (to be ported) » Fixed

@smustgrave Because these random failures also impair the security advisory process, which is currently provided back to 9.4.x.

spokje’s picture

Are we sure we want to mark this as fixed?
Doesn't seem it has landed anywhere else but in 10.1.x.

catch’s picture

Version: 10.1.x-dev » 10.0.x-dev
Status: Fixed » Patch (to be ported)

Yes this still need backport, but the original commit should cherry-pick I think.

xjm’s picture

Version: 10.0.x-dev » 9.4.x-dev

Sorry, had the issue open in two tabs and crossposted.

Removing credit for the unnecessary backport patch.

  • catch committed 1778d192 on 10.0.x
    Issue #3353096 by Spokje, catch, xjm: [random test failure] Try to un-...

  • catch committed e3c102c7 on 9.5.x
    Issue #3353096 by Spokje, catch, xjm: [random test failure] Try to un-...
catch’s picture

Title: [backport to 9.4.x] [random test failure] Try to un-skip and fix InlineBlockPrivateFilesTest::testPrivateFiles() in context of [#3353085] » [random test failure] Try to un-skip and fix InlineBlockPrivateFilesTest::testPrivateFiles() in context of [#3353085]
Version: 9.4.x-dev » 9.5.x-dev
Status: Patch (to be ported) » Fixed

I agree with backporting the original bugfix back to 9.4.x since that will fix random test failures there. With these patches though we're only unskipping previously-skipped tests, some/most of which were only skipped in the first place in 9.5.x or later, so I think it's better to stop at 9.5. We wouldn't backport new test coverage to 9.4.x at this point and this is similar.

I want to get this round of fixes all in since it's taken weeks to commit them 12-24 hours apart in 10.1.x, so going ahead with the cherry-picks to 10.0.x and 9.5.x but leaving there. @xjm if you strongly think they should go back to 9.4.x too please re-open although also we should double check they're skipped in the first place on that branch.

Made the changes directly in the 10.0.x branch since there were two commits on this issue and cherry-picked that commit to 9.5.x, thanks all!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.