Problem/Motivation

1) Drupal\Tests\views\FunctionalJavascript\Plugin\views\Handler\FilterTest::testAddingFilter
The final count was: 2
Failed asserting that false is true.

/var/www/html/vendor/phpunit/phpunit/src/Framework/Constraint/Constraint.php:121
/var/www/html/vendor/phpunit/phpunit/src/Framework/Constraint/Constraint.php:55
/var/www/html/core/modules/views/tests/src/FunctionalJavascript/Plugin/views/Handler/FilterTest.php:79
/var/www/html/vendor/phpunit/phpunit/src/Framework/TestResult.php:726

Steps to reproduce

Proposed resolution

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

Comments

alexpott created an issue. See original summary.

alexpott’s picture

Status: Active » Needs review
StatusFileSize
new2.94 KB
new5.05 KB

The last submitted patch, 2: 3315227-2-test-only.patch, failed testing. View results

Status: Needs review » Needs work

The last submitted patch, 2: 3315227-2.patch, failed testing. View results

alexpott’s picture

Status: Needs work » Needs review
StatusFileSize
new5.39 KB

Status: Needs review » Needs work

The last submitted patch, 5: 3315227-5.patch, failed testing. View results

alexpott’s picture

Status: Needs work » Needs review
StatusFileSize
new5.66 KB
alexpott’s picture

StatusFileSize
new5.64 KB

Status: Needs review » Needs work

The last submitted patch, 8: 3315227-8.patch, failed testing. View results

alexpott’s picture

Status: Needs work » Needs review
StatusFileSize
new5.6 KB

So it was the other sleep that fixed it... interesting...

alexpott’s picture

StatusFileSize
new5.67 KB

I wonder if this works...

Status: Needs review » Needs work

The last submitted patch, 11: 3315227-11.patch, failed testing. View results

alexpott’s picture

Status: Needs work » Needs review
StatusFileSize
new5.66 KB
alexpott’s picture

StatusFileSize
new4.56 KB

Let's see if only 10ms wait works.

Status: Needs review » Needs work

The last submitted patch, 14: 3315227-14.patch, failed testing. View results

alexpott’s picture

Status: Needs work » Needs review
StatusFileSize
new4.56 KB

Status: Needs review » Needs work

The last submitted patch, 16: 3315227-16.patch, failed testing. View results

alexpott’s picture

Status: Needs work » Needs review
StatusFileSize
new4.55 KB
alexpott’s picture

StatusFileSize
new1.42 KB

here's the proper ptch

alexpott’s picture

StatusFileSize
new4.22 KB

Let's make sure we can have the smallest possible change.

alexpott’s picture

StatusFileSize
new1.08 KB

And now the patch for realz...

andypost’s picture

+++ b/core/modules/views/tests/src/FunctionalJavascript/Plugin/views/Handler/FilterTest.php
@@ -121,9 +121,11 @@ protected function filterVisibleElements($elements) {
-    return $page->waitFor($timeout / 1000, function () use ($count, $page, $locator) {
+    return $page->waitFor($timeout / 1000, function () use ($count, $page, $locator, &$final_count) {

not clear where $final_count is used, moreover it's not defined in the function body

alexpott’s picture

StatusFileSize
new837 bytes

@andypost that was old debug code I removed the other parts of it but forgot that. Ooops.

alexpott’s picture

StatusFileSize
new4.09 KB

I think I've worked out why this is happening! This is a slowness issue. We never do the check twice because things are going slow.. the callback in the wait here makes a tonne of requests for each td to find out if it is displayed...

Here's an implementation that I think will check twice.

alexpott’s picture

StatusFileSize
new981 bytes

Here's the final patch (hopefully)... I'm running the entire JS test suite with this change over on #3315074-15: What's going on with JS testing discovery issue

The last submitted patch, 21: 3315227-21.patch, failed testing. View results

alexpott’s picture

StatusFileSize
new5.41 KB

Another idea based on old fixes...

Status: Needs review » Needs work

The last submitted patch, 27: 3315227-26.patch, failed testing. View results

alexpott’s picture

Status: Needs work » Needs review
StatusFileSize
new5.4 KB

Whoops miessed up the timeouts...

The last submitted patch, 19: 3315227-19.patch, failed testing. View results

The last submitted patch, 25: 3315227-25.patch, failed testing. View results

Status: Needs review » Needs work

The last submitted patch, 29: 3315227-28.patch, failed testing. View results

alexpott’s picture

Status: Needs work » Needs review
StatusFileSize
new5.74 KB
alexpott’s picture

StatusFileSize
new2.61 KB

So I think we should consider committing this. It moves the wait fixes from #3211164: Random errors in Javascript Testing up a level so more code benefits and it ensures the callback always fires twice regardless of how long the callback takes.

spokje’s picture

Annoying nit alert:

--- a/core/tests/Drupal/Tests/DocumentElement.php
+++ b/core/tests/Drupal/Tests/DocumentElement.php
@@ -85,4 +88,31 @@ public function getText() {

+      // If the callback has only fired once then in might be really slow. In

+ // If the callback has only fired once then it

alexpott’s picture

StatusFileSize
new946 bytes
new2.62 KB

Fixed and improved the comment.

The last submitted patch, 34: 3315227-34.patch, failed testing. View results

quietone’s picture

In #34, the failure is with JSWebWithWebDriverAssertTest::testJsWebAssert. That is not listed on #2829040: [meta] Known intermittent, random, and environment-specific test failures so I was wondering if a new random error was being introduced. I searched for that test and find that it did fail in the most recent batch of random failures that occurred 21 and 22 July of this year. It is also failing random in #3315074-21: What's going on with JS testing discovery issue. So, not a new random error.

This is a definitive improvement!

spokje’s picture

Status: Needs review » Reviewed & tested by the community

JSWebAssertTest and JSWebWithWebDriverAssertTest still seem to have issues, but this is overall such an improvement on the current situation that I'm going to RTBC.

The troubles there seem to be based on a test that asserts if a element that should dissappear after X is still there before X has passed.
I think that's kinda a niche case where waitFor could still fail, but that's for the next issue IMHO.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 36: 3315227-36.patch, failed testing. View results

spokje’s picture

Status: Needs work » Reviewed & tested by the community
andypost’s picture

Status: Reviewed & tested by the community » Needs work

Not sure it makes sense to commit patch which does not pass tests, the issue here is because of runner's performance and DA could move to faster instance

catch’s picture

Status: Needs work » Reviewed & tested by the community

The test suite shouldn't depend on how fast the test runners are though, I think we need to get this in, which should result in green along with the other change in #3315074: What's going on with JS testing discovery issue.

  • catch committed c3322d4 on 10.0.x
    Issue #3315227 by alexpott, Spokje, andypost, quietone: Drupal\Tests\...
  • catch committed eb8c8d4 on 10.1.x
    Issue #3315227 by alexpott, Spokje, andypost, quietone: Drupal\Tests\...
  • catch committed 164c603 on 9.4.x
    Issue #3315227 by alexpott, Spokje, andypost, quietone: Drupal\Tests\...
  • catch committed 196e873 on 9.5.x
    Issue #3315227 by alexpott, Spokje, andypost, quietone: Drupal\Tests\...
catch’s picture

Version: 10.1.x-dev » 9.4.x-dev
Status: Reviewed & tested by the community » Fixed

Committed/pushed to 10.1.x, cherry-picked back through to 9.4.x, thanks!

spokje’s picture

@andypost: I see what you're saying, but (at least for me) there's a difference between not green and less red.

The current JS test fail-storm is rather big. Most probably, as you said, caused by upped performance on our test-CI (damn you, progress!!!).

Looking for One-Patch-To-Rule/Greenify-Them-All, might be one step beyond, by chopping it into smaller fixes (that will still not be fully green) we might be able to get to the Goal of Green quicker.

INSTA-EDIT: Outtyped _and_ outcommitted, I'm getting too slow for this game...;)

alexpott’s picture

@Spokje the current storm of fails is not caused by more performance. The results here show that the currently the testbot often can't call the callback here twice in 10 seconds! Whereas locally this is never an issue. Something happened to test performance on Oct 11 - not sure what and even if we're are in control of the change. There ate discussions to use a different instance type that hopefully will result in faster tests and less random fails.

spokje’s picture

@alexpott: Ah, misinterpreted the performance the wrong way around. Thanks for clearing that up.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.