Previously on "Random Fails":

Today:

Drupal\Tests\outside_in\FunctionalJavascript\OutsideInBlockFormTest::testBlocks with data set "block-powered" ('system_powered_by_block', 'Can you imagine anyone showin... block', 'span a', 'h2', 'Save Powered by Drupal', '#toolbar-item-user')
Failed asserting that a NULL is not empty.

/var/www/html/core/modules/outside_in/tests/src/FunctionalJavascript/OutsideInJavascriptTestBase.php:143

Examples:

CommentFileSizeAuthor
#13 2901626-13.patch1.19 KBtedbow
#13 interdiff-10-13.txt1.05 KBtedbow
#10 2901626-10.patch1.16 KBtedbow
#6 2848177-14.patch1.16 KBAnonymous (not verified)
#5 2901626-5-x30-no_transitions.patch1.71 KBtedbow
#5 2901626-5-x30-no_changes.patch563 bytestedbow
#4 x30__wait-1_2901626-4-test-only.patch1.29 KBAnonymous (not verified)
#2 x200__wait-1_2901626-2-test-only.patch1.51 KBAnonymous (not verified)

Comments

Anonymous’s picture

Anonymous (not verified) commented

vaplas created an issue. See original summary.

Anonymous’s picture

Anonymous (not verified) commented
Status: Active » Needs review
StatusFileSize
new x200__wait-1_2901626-2-test-only.patch1.51 KB

Status: Needs review » Needs work

The last submitted patch, 2: x200__wait-1_2901626-2-test-only.patch, failed testing. View results

Anonymous’s picture

Anonymous (not verified) commented
Status: Needs work » Needs review
StatusFileSize
new x30__wait-1_2901626-4-test-only.patch1.29 KB

I confused the place of error. It seems this is what was said here #2848177-14: More \Drupal\Tests\outside_in\FunctionalJavascript\OutsideInBlockFormTest random fails.

tedbow’s picture

tedbow
he/him
Primary language English
Location Ithaca, NY, USA
commented
StatusFileSize
new 2901626-5-x30-no_changes.patch563 bytes
new 2901626-5-x30-no_transitions.patch1.71 KB

Ok I was working with @drpal the yesterday and we were seeing similar random fail on this local.

We fixed it by removing all animations for testing. Could this a solution? We don't actually test anything in relation to animation transitions in our tests. Why do we need them?

This patch adds removing all transitions to core/modules/outside_in/tests/modules/outside_in_test_css/css/css_fix.theme.css which will already use because our testing doesn't work pointer events CSS.

I am also uploading at patch with only changes being the running the test 30 times. To test if we are likely to get a random fail.

Anonymous’s picture

Anonymous (not verified) commented
StatusFileSize
new 2848177-14.patch1.16 KB

Constantly worrying about turning off the animation for a stable passing of tests seems a little tedious, although it probably makes sense.
What if we add a patch from #2848177-14: More \Drupal\Tests\outside_in\FunctionalJavascript\OutsideInBlockFormTest random fails?

Anonymous’s picture

Anonymous (not verified) commented

I overlooked that all transitions were canceled globally. Good idea! Enhanced testing was launched.

Anonymous’s picture

Anonymous (not verified) commented

Well, #5 looks nice. I created another issue to solve the problem of stability tests due to CSS rules. So, we can add it like @todo for current solution, or not. In any case, +1 to RTBC with #5. This will help to quickly solve random fails of tests such types. Thanks!

Anonymous’s picture

Anonymous (not verified) commented
Related issues: +#2901792: Disable all animations in Javascript testing
tedbow’s picture

tedbow
he/him
Primary language English
Location Ithaca, NY, USA
commented
StatusFileSize
new 2901626-10.patch1.16 KB

Since @vaplas +1 rtbc for #5 here is the patch with only the needed test change

We also already have follow to see if there is better way to test animations #2901792: Disable all animations in Javascript testing

tim.plunkett’s picture

tim.plunkett
he/him
Primary language English
Location Philadelphia
commented 
+++ b/core/modules/outside_in/tests/modules/outside_in_test_css/css/css_fix.theme.css
@@ -2,3 +2,25 @@
+  /*CSS transforms*/
...
+  /*CSS animations*/

These two comments should have whitespace before and after the words. And maybe punctuation?
And it leads me to wonder what exactly the first section is, can it get a comment too?

tedbow’s picture

tedbow
he/him
Primary language English
Location Ithaca, NY, USA
commented
Title: The Returned of the \Drupal\Tests\outside_in\FunctionalJavascript\OutsideInBlockFormTest random fails » CSS animations cause \Drupal\Tests\outside_in\FunctionalJavascript\OutsideInBlockFormTest random fails
tedbow’s picture

tedbow
he/him
Primary language English
Location Ithaca, NY, USA
commented
StatusFileSize
new interdiff-10-13.txt1.05 KB
new 2901626-13.patch1.19 KB

#11 fixed

grandmaglassesropeman’s picture

grandmaglassesropeman
he/him
Primary language English
Location 🌎
commented
Status: Needs review » Reviewed & tested by the community

This looks good, and probably needed until we can migrate away from Phantom.

🚀

catch’s picture

catch
he/him
Primary language English
commented
Version: 8.5.x-dev » 8.4.x-dev
Status: Reviewed & tested by the community » Fixed

Committed/pushed to 8.5.x and cherry-picked to 8.4.x. Thanks!

  • xjm committed b3cd6ee on 8.4.x 
    Issue #2901626 by tedbow, vaplas: CSS animations cause \Drupal\Tests\...

  • xjm committed 96de2d9 on 8.5.x 
    Issue #2901626 by tedbow, vaplas: CSS animations cause \Drupal\Tests\...
xjm’s picture

xjm
she/her
Primary language English
commented

🚀

tacituseu’s picture

tacituseu commented
Parent issue: » #2829040: [meta] Known intermittent, random, and environment-specific test failures

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

tedbow’s picture

tedbow
he/him
Primary language English
Location Ithaca, NY, USA
commented
Component: outside_in.module » settings_tray.module
xjm’s picture

xjm
she/her
Primary language English
commented
Issue tags: +Random test failure
Parent issue: #2829040: [meta] Known intermittent, random, and environment-specific test failures »