Problem/Motivation

On March 5 and 6 (following the Chromedriver update), numerous Functional JavaScript tests are failing more frequently than they have previously. It's basically "every core HEAD test run fails for all branches and environments, several commits in a row", with each job failing one or more of the tests in this patch. This basically halted all core development for an entire day.

Several of the test failures have recurred this evening. Some of them are tests that have been in core for years and never once had a random fail before.

Examples of fails

Steps to reproduce

Look at https://www.drupal.org/node/3060/qa. Screenshot from March 5:

Proposed resolution

Skip tests that failed in HEAD three or more times in the past day, then get each working again in a followup.

Remaining tasks

We can add these tests to the scope of #3267247: [meta] Fix and re-enable tests skipped for random failures, although since none of the tests that were already skipped are magically working again, I'd prefer to turn that into a meta and debug the tests individually per method.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

xjm created an issue. See original summary.

xjm’s picture

Version: 9.4.x-dev » 9.3.x-dev
FileSize
4.62 KB
xjm’s picture

Status: Active » Needs review
xjm’s picture

Issue summary: View changes
xjm’s picture

xjm’s picture

Issue summary: View changes
lauriii’s picture

Status: Needs review » Reviewed & tested by the community

More test cases to deal with in #3267247: [meta] Fix and re-enable tests skipped for random failures I guess but I think this is still the right thing to do since this is definitely having an impact on productivity of core development.

  • lauriii committed dd31b18 on 10.0.x
    Issue #3268070 by xjm: Temporarily skip even more failing tests
    

  • lauriii committed 71d0234 on 9.4.x
    Issue #3268070 by xjm: Temporarily skip even more failing tests
    
    (cherry...

  • lauriii committed df32cec on 9.3.x
    Issue #3268070 by xjm: Temporarily skip even more failing tests
    
    (cherry...
lauriii’s picture

Status: Reviewed & tested by the community » Fixed

Committed #2 to 10.0.x, 9.4.x and 9.3.x to get the tests passing again.

xjm’s picture

Version: 9.3.x-dev » 9.2.x-dev
Status: Fixed » Reviewed & tested by the community
FileSize
3.93 KB

Here's a 9.2.x version (minus the skip for MediaLibraryTest because it doesn't exist in 9.2.x, and is already fixed anyway).

I checked over the list to make sure none of these tests were already fixed in other commits. Only CKE5 MediaLibraryTest and AjaxTest are already fixed.

  • xjm committed ffe8c20 on 9.2.x
    Issue #3268070 by xjm, lauriii: Temporarily skip even more failing tests
    
xjm’s picture

Status: Reviewed & tested by the community » Fixed

Committed to 9.2.x.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.