Problem/Motivation

https://git.drupalcode.org/issue/drupal-2921627/-/jobs/7892732#L479

Media (Drupal\Tests\ckeditor5\FunctionalJavascript\Media)
     ✔ Drupal media
     ✔ Media arbitrary html
     ✔ Alt disabled
     ✘ Translation alt
       ┐
       ├ Behat\Mink\Exception\ElementNotFoundException: Element matching xpath "//img[contains(@alt, "Zartan est le chef des Dreadnoks.")]" not found.
       │
       │ /builds/issue/drupal-2921627/vendor/behat/mink/src/WebAssert.php:465
       │ /builds/issue/drupal-2921627/core/modules/ckeditor5/tests/src/FunctionalJavascript/MediaTest.php:585
       ┴
     ✔ Drupal media style in dropdown
     ✔ Drupal media style with class
     ✔ View mode with with·alignment
     ✔ View mode with without·alignment
    
    FAILURES!
    Tests: 8, Assertions: 315, Failures: 1.

Steps to reproduce

Baseline MR with only a comment change:
https://git.drupalcode.org/project/drupal/-/merge_requests/14331

Proposed resolution

Use waitForElement() instead of directly calling elementExists() after pressButton().

https://git.drupalcode.org/project/drupal/-/merge_requests/14332

Remaining tasks

  1. Verify results of baseline vs. fix repeat jobs
  2. Reviews / refinements
  3. RTBC
  4. Commit / rejoice

User interface changes

Introduced terminology

API changes

Data model changes

Release notes snippet

Issue fork drupal-3566844

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

kentr created an issue. See original summary.

dww made their first commit to this issue’s fork.

dww’s picture

dww
we/he/they
commented
Assigned: Unassigned » dww
Issue summary: View changes
Issue tags: +Bug Smash Initiative

Working on it...

dww opened merge request !14331

dww opened merge request !14332

dww’s picture

dww
we/he/they
commented
Issue summary: View changes
Status: Active » Needs work

MRs open. Repeat jobs running. Leaving NW until the repeat results are done and I can update the summary accordingly.

dww’s picture

dww
we/he/they
commented
Issue summary: View changes

First set of repeat jobs (100x) passed on both MRs. Trying again at 300x to try to get the random fail to fail.

dww’s picture

dww
we/he/they
commented
Issue summary: View changes

300x is too many, this test takes so long, that timed out after around 250 runs in both MRs. Trying again to trigger failures, this time with a pair of 200x runs.

dww’s picture

dww
we/he/they
commented
Assigned: dww » Unassigned
Issue summary: View changes
Status: Needs work » Needs review

Phew, 3rd time was the charm. Finally caught the baseline red handed with 2 fails out of 200:
https://git.drupalcode.org/issue/drupal-3566844/-/jobs/7982078

Ready for review.

mstrelan’s picture

mstrelan commented
Status: Needs review » Reviewed & tested by the community

The change looks good and the format of the assertion matches what we're doing before the save button is pressed. The only observation I'll make is that the baseline passed 548/550 runs and the fix passed 554/554. The margin of error is so slim that it's entirely possible this is only passing with a bit of luck, but I don't think this is the case here.

dww’s picture

dww
we/he/they
commented

Thanks!

Happy to do more repeat runs to see if the ratio improves and to confirm the fix never fails. Not sure it's worth the CO2 and $.

  • catch committed b686f2b9 on 11.3.x 
    fix: #3566844 [random test failure] MediaTest::testTranslationAlt

By:...

  • catch committed d5c8096d on 11.x 
    fix: #3566844 [random test failure] MediaTest::testTranslationAlt

By:...
catch’s picture

catch
he/him
Primary language English
commented
Version: 11.x-dev » 11.3.x-dev
Status: Reviewed & tested by the community » Fixed

There have been some random failures where it was a really strange race condition and repeat runs showed we didn't actually fix it, but so many random failures in js tests are due to not waiting for an element to appear I think we can just go ahead here.

Committed/pushed to 11.x and cherry-picked to 11.3.x, thanks!

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.

catch closed merge request !14332

dww closed merge request !14331

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.