Problem/Motivation
There are many core issues where JavaScript tests have gone awry because they expect certain conditions to be true (i.e., certain text or elements to be present) when they are not. Usually these tests try to use JsWebAssert::assertWaitOnAjaxRequest() as a way to wait for the browser to get things in the right state, but that method seems to be quite unreliable.
A working theory that myself, @bnjmnm, and @alexpott have come up with is that, although assertWaitOnAjaxRequest() waits for jQuery to complete AJAXing and animating, it does not have verify that any code in the AJAX command callbacks (i.e., anything in Drupal.AjaxCommands.prototype) is not running. (That's one example that springs to mind, but there could be others.) These kinds of small timing differences may be the reason for the unreliability.
Child issues:
- #3102990: FormGroupingElementsTest::testVerticalTabChildVisibility passing erroneously
- #3102981: Invalid selector in EditModeTest
- #3103034: Invalid selector in MaximumFileSizeExceededUploadTest
- #3103101: Make the JsWebAssert::waitFor*() methods behave like real assertions
- #3103081: Deprecate JsWebAssert::assertWaitOnAjaxRequest() in Drupal 9.0 and Drupal 8.9 and Replace usages of JsWebAssert::assertWaitOnAjaxRequest() in core
Optional:
- #3103096: Move waitForElementsCount from MediaLibraryTestBase to JSWebAssert
- #3103097: Move waitForNoText from MediaLibraryTestBase to JSWebAssert
Proposed resolution
Deprecate assertWaitOnAjaxRequest() and change the waitFor*() methods in JsWebAssert so that they actually assert their expectations, rather than returning boolean status values. This is also much more developer-friendly because it allows JavaScript tests to assert what they actually expect, rather than trying to hack into and around the low-level workings of whatever is happening in the browser.
Remaining tasks
That thing.
User interface changes
None.
API changes
A widely-used JavaScript testing method will be deprecated, and other assertion methods will change behavior slightly.
Data model changes
None.
Release notes snippet
TBD
| Comment | File | Size | Author |
|---|---|---|---|
| #46 | 3061852-46.patch | 193.82 KB | oknate |
| #46 | 3061852--interdiff-43-46.txt | 1.29 KB | oknate |
| #43 | 3061852-43.patch | 192.91 KB | oknate |
| #43 | 3061852--interdiff-42-43.txt | 21.38 KB | oknate |
| #42 | 3061852-42.patch | 198.4 KB | oknate |
Comments
Comment #2
phenaproximaAdding #1988968: Drupal.ajax does not guarantee that "add new JS file to page" commands have finished before calling said JS as a related issue.
Comment #3
phenaproximaOh, and #3055648: Frequent random fail in \Drupal\Tests\media_library\FunctionalJavascript\MediaLibraryTest.
Comment #4
phenaproximaComment #5
phenaproximaComment #6
wim leersI pointed @phenaproxima to #1988968: Drupal.ajax does not guarantee that "add new JS file to page" commands have finished before calling said JS. If we can confirm that this is is caused by that, perhaps we can finally get that issue fixed :) OTOH it's already marked and I don't think it can possibly meet the requirements for .
Comment #7
phenaproximaA little something to get the ball rolling.
Comment #8
phenaproximaA little clean-up, and fixing utterly wrong and incorrect doc comment on JsWebAssert::waitForText().
Comment #9
oknateNice. Could get a
waitForTextRemoved()method too?@bnjmn is using one for #3055648: Frequent random fail in \Drupal\Tests\media_library\FunctionalJavascript\MediaLibraryTest
He calls it
WaitForNoText(),but I think to be more consistentwaitForTextRemoved()is better.Or should that be a different issue?
Comment #10
phenaproximaI think that's out of scope here. Adding new assertions should be done in a separate issue(s); this is just about improving the ones we already have.
Comment #11
oknatePerhaps it's an antipattern, but I have used
$this->assertEmpty($assert_session-> waitForElementVisible('css', '.something', 1000)); The change to waitForElementVisible() will break those types of assertions. What should they be replaced with?I guess waitForElementRemoved(), which is being added in 8.8?
Comment #12
phenaproximaFor those cases, we'd need to add a new assertion (waitForElementNotVisible()) or similar. Presumably, those kinds of breakages will be revealed by this patch.
Comment #13
oknateI think
waitForElementRemoved():https://www.drupal.org/node/2906685
Comment #17
oknateRemoving most (if not all) of the
$this->assertNotEmpty()and$this->assertNotNull()code wrapped around JsWebAssert::waitFor*() methods.Comment #18
oknateComment #20
oknateBroke off a side issue for one of the failures: #3102981: Invalid selector in EditModeTest
Comment #21
oknateFixing a test or two, fixing some missed assertions testing the result of the updated methods which need to be removed or replaced.
Comment #22
oknateAdding another side issue: #3102990: FormGroupingElementsTest::testVerticalTabChildVisibility passing erroneously
Comment #23
oknatePostponing on
We can't really fix this one until those are fixed. But we should keep going through the failures here to see if there are similar issues where tests were passing erroneously.
Comment #25
oknateComment #27
oknatemore fixes
Comment #28
oknatemaking method public
Comment #30
oknaterestoring one of the methods removed from the MediaLibraryTestBase.
Comment #31
oknatemaking a method public, same mistake as #28 on a different method.
Comment #32
oknateBreaking off one part of this as a separate issue: #3103034: Invalid selector in MaximumFileSizeExceededUploadTest
I didn't include it in the file name, because I just created it.
Also, I checked and there are 240 occurrences of assertWaitOnAjaxRequest() in core. Given that this issue is already large, I think we should come up with a way of splitting this up.
Here's how we might do that:
Another thing we could do is make this a META issue.
Comment #33
oknateComment #34
oknateFixing the failure, and marking as postponed.
Comment #35
oknateI think (IMHO), this should be a META issue, and we can do it as a series of child issues.
Comment #36
oknateComment #37
oknateComment #38
oknateComment #39
oknateComment #40
oknateComment #41
oknateDeprecating
assertWaitOnAjaxRequest().Update: I guess this isn't enough, we'll need to add a @trigger_error: https://www.drupal.org/core/deprecation.
Comment #42
oknateAdding
@trigger_errortoassertWaitOnAjaxRequest().Comment #43
oknate1) In #25 I moved some methods from MediaLibraryTestBase to JsWebAssert. I think some of that was out of scope, so reverting those changes. I opened two separate issue to explore moving two of these methods to JsWebAssert:
2) The methods in JsWebAssert to which we added assertions, if they were returning a boolean, no longer need a return. If they fail, they'll throw an error, so there's no point of a return value, AFAIK. They either pass or throw an error. Those that return a node element can still return it, in case it's needed, but the documentation should be updated to indicate it won't return NULL, but with throw an error, for example:
Comment #44
oknateComment #45
oknateComment #46
oknateUpdating the @todo items based on #43 to refer to those two new issues.
Comment #47
oknateComment #48
oknateComment #49
oknateComment #50
oknateCreated a subtask for the work related to updating the assertions: #3103101: Make the JsWebAssert::waitFor*() methods behave like real assertions
This leaves this issue as a META issue. It also allows 3103101 to be committed independent of #3103081: Deprecate JsWebAssert::assertWaitOnAjaxRequest() in Drupal 9.0 and Drupal 8.9 and Replace usages of JsWebAssert::assertWaitOnAjaxRequest() in core, which will take a lot more work to be ready to commit.
Comment #51
oknateComment #55
tim.plunkettOpened #3230726: JsWebAssert::assertWaitOnAjaxRequest() should use Behat\Mink\Exception\ExpectationException not RuntimeException as a short-term solution for problems with assertWaitOnAjaxRequest
Comment #58
nod_Because of the new add_js command, ajax.ajaxing will now wait for the commands to finish executing before being set to false. That should help a bit.