Problem/Motivation

 PHPUnit Test failed to complete; Error: PHPUnit 9.6.8 by Sebastian Bergmann
    and contributors.
    
    Testing Drupal\FunctionalJavascriptTests\Ajax\ThrobberTest
    .E                                                                  2 / 2
    (100%)
    
    Time: 00:37.165, Memory: 4.00 MB
    
    There was 1 error:
    
    1)
    Drupal\FunctionalJavascriptTests\Ajax\ThrobberTest::testProgressThrobberPosition
    Behat\Mink\Exception\ElementNotFoundException: Element matching xpath
    "//div[contains(@class,
    "dropbutton-wrapper")]/following-sibling::div[contains(@class,
    "ajax-progress-throbber")]" not found.
    
    /builds/project/drupal/vendor/behat/mink/src/WebAssert.php:418
    /builds/project/drupal/core/tests/Drupal/FunctionalJavascriptTests/Ajax/ThrobberTest.php:109
    /builds/project/drupal/vendor/phpunit/phpunit/src/Framework/TestResult.php:728
    
    ERRORS!
    Tests: 2, Assertions: 18, Errors: 1.

Steps to reproduce

Proposed resolution

- Wait for elements that are more specific than (an already present) link text.
- Place hold_test_response at the appropriate position.
- Don't use root user just because it "conveniently" has all permissions.
- Don't use a separate method to prevent a full init and destroy of a testsite for testing one throbber.

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

Issue fork drupal-3387772

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

Spokje created an issue. See original summary.

spokje’s picture

Status: Active » Needs review
smustgrave’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: +Needs Review Queue Initiative

Seems like a simple update and didn't break anything!

catch’s picture

Status: Reviewed & tested by the community » Needs review

This seems good, I'm also seeing random test failures in HEAD on gitlab for this test: https://git.drupalcode.org/project/drupal/-/jobs/113782

Including the one on the pipeline for this MR: https://git.drupalcode.org/issue/drupal-3387772/-/jobs/93792

We probably need to fix that at the same time as this, otherwise it's hard to show this issue hasn't introduced a new failure.

spokje’s picture

Title: Remove unneeded login from ThrobberTest::testProgressThrobberPosition by refactoring setUp() away » Random testfailures in ThrobberTest::testProgressThrobberPosition by refactoring setUp() away
Assigned: Unassigned » spokje
Status: Needs review » Needs work
Issue tags: +Random test failure
spokje’s picture

Issue summary: View changes

spokje’s picture

Issue summary: View changes
spokje’s picture

Issue summary: View changes
spokje’s picture

Title: Random testfailures in ThrobberTest::testProgressThrobberPosition by refactoring setUp() away » [random test failure] Random failure in ThrobberTest::testProgressThrobberPosition by refactoring setUp() away
Category: Task » Bug report
StatusFileSize
new2 KB
spokje’s picture

Issue summary: View changes
spokje’s picture

StatusFileSize
new2 KB
new4.6 KB

Well, that wasn't the most "inspired" test I've encountered.
Extra joy-bonus for 100% failing to pass locally.

Anyway, let's see if this is any better.

spokje’s picture

One (per usual) SQLSTATE[HY000]: General error: 5 database is locked exception in the SQLite run in 10.000 runs vs 3 actual failures in 5000 runs of the current test.

Actual fix in the MR.

spokje’s picture

Assigned: spokje » Unassigned
Status: Needs work » Needs review
spokje’s picture

Title: [random test failure] Random failure in ThrobberTest::testProgressThrobberPosition by refactoring setUp() away » [random test failure] Random failure in ThrobberTest::testProgressThrobberPosition
spokje’s picture

spokje’s picture

Issue summary: View changes
smustgrave’s picture

Open thread and then can mark.

spokje’s picture

Assigned: Unassigned » spokje
Status: Needs review » Needs work

Going to change some stuff and answer thread.

spokje’s picture

Issue summary: View changes
spokje’s picture

StatusFileSize
new2 KB
new4.63 KB

Moved dropbutton throbber test to the existing test to prevent an extra setup/teardown.

Testing new solution

spokje’s picture

spokje’s picture

Works, besides the usual SQLite locked DB failure, also seems quite a bit faster:

- Old on PHP 8.2/MySQL 8 1000x

22 min build duration;

- New 2000x

24 min build duration;
spokje’s picture

Status: Needs work » Needs review

Answering the thread from @catch:

One question - is this hold_test_response() stuff still necessary after the other changes? It's only used in this test and MediaLibraryTestBase now.

Yes, we do.

We're testing AJAX behaviour _before_ the response arrives and the throbber/whatever is replaced with the actual response.
Somehow hold_test_response works, due to delaying the response by writing to a file it seems.

Most of the time we're "saved" by the slowness of Drupal CI and the test is finished before the response arrives. However GitLab seems a tad quicker, thus more random failures on testing this stuff.

The whole methodology is besides non-documented not really deterministic, opened #3390641: Refactor AJAX-testing hold_test test module to do something more along the lines we did in #3375959: Add a way to delay executions in test runner until terminate event completed in the child site.

TLDR: Yes, we need it now and in the future, but we need a better way to temporary delay responses to be able to test "inbetween AJAX response-time" behaviour.

smustgrave’s picture

Status: Needs review » Reviewed & tested by the community

All threads appear to be addressed.

catch’s picture

Status: Reviewed & tested by the community » Fixed

Committed/pushed to 11.x and cherry-picked to 10.1.x, thanks!

  • catch committed 94e8a471 on 10.1.x
    Issue #3387772 by Spokje: [random test failure] Random failure in...

  • catch committed 6da04293 on 11.x
    Issue #3387772 by Spokje: [random test failure] Random failure in...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.