Problem/Motivation
On March 5 and 6 (following the Chromedriver update), numerous Functional JavaScript tests are failing more frequently than they have previously. It's basically "every core HEAD test run fails for all branches and environments, several commits in a row", with each job failing one or more of the tests in this patch. This basically halted all core development for an entire day.
Several of the test failures have recurred this evening. Some of them are tests that have been in core for years and never once had a random fail before.
Examples of fails
LayoutBuilderNestedFormUiTest::testAddingFormBlocksToOverrides()
: https://www.drupal.org/pift-ci-job/2333207LayoutBuilderTest::testLayoutBuilderUi()
andLayoutBuilderNestedFormUiTest::testAddingFormBlocksToDefaults()
: https://www.drupal.org/pift-ci-job/2333660BlockFormMessagesTest::testValidationMessage()
(and the above): https://www.drupal.org/pift-ci-job/2333662ContentPreviewToggleTest::testContentPreviewToggle()
andLayoutBuilderDisableInteractionsTest::testFormsLinksDisabled()
: https://www.drupal.org/pift-ci-job/2333666MediaLibraryTest::testButton()
https://www.drupal.org/pift-ci-job/2333668QuickEditIntegrationTest::testArticleNode()
: https://www.drupal.org/pift-ci-job/2333161
Steps to reproduce
Look at https://www.drupal.org/node/3060/qa. Screenshot from March 5:
Proposed resolution
Skip tests that failed in HEAD three or more times in the past day, then get each working again in a followup.
Remaining tasks
We can add these tests to the scope of #3267247: [meta] Fix and re-enable tests skipped for random failures, although since none of the tests that were already skipped are magically working again, I'd prefer to turn that into a meta and debug the tests individually per method.
Comment | File | Size | Author |
---|---|---|---|
#12 | skip-more-tests-3268070-9.2.x.patch | 3.93 KB | xjm |
#2 | skip-more-tests.patch | 4.62 KB | xjm |
| |||
Drupal_core___Drupal_org.png | 331.62 KB | xjm |
Comments
Comment #2
xjmComment #3
xjmComment #4
xjmComment #5
xjmComment #6
xjmComment #7
lauriiiMore test cases to deal with in #3267247: [meta] Fix and re-enable tests skipped for random failures I guess but I think this is still the right thing to do since this is definitely having an impact on productivity of core development.
Comment #11
lauriiiCommitted #2 to 10.0.x, 9.4.x and 9.3.x to get the tests passing again.
Comment #12
xjmHere's a 9.2.x version (minus the skip for
MediaLibraryTest
because it doesn't exist in 9.2.x, and is already fixed anyway).I checked over the list to make sure none of these tests were already fixed in other commits. Only CKE5
MediaLibraryTest
andAjaxTest
are already fixed.Comment #14
xjmCommitted to 9.2.x.