Problem/Motivation
Since #3091870: Fail Functional Javascript tests that throw Javascript errors all JavaScript errors thrown during test execution are supposed to fail the test run. However, the JavaScript error produced by patch from #3305706-30: Uncaught TypeError: this.$pluginSelect.find(...)[0] is undefined. Does not fail the test.
Steps to reproduce
- Apply patch from #3305706-30: Uncaught TypeError: this.$pluginSelect.find(...)[0] is undefined.
- Ensure that test run is paused after the
dragTo()call inEntityDisplayTest::testHiddenExtraFields()(using a breakpoint or a strategically placed wait as recommended in the docs.) - Run the test:
./vendor/bin/phpunit -v --filter testHiddenExtraFields core/modules/field_ui/tests/src/FunctionalJavascript/EntityDisplayTest.php - Inspect the browser console. It will should show the following error:
field_ui.js?v=10.1.0-dev:371 Uncaught TypeError: Cannot read properties of undefined (reading 'value') at Drupal.fieldUIDisplayOverview.field.regionChange (field_ui.js?v=10.1.0-dev:371:51) at Drupal.tableDrag.onDrop (field_ui.js?v=10.1.0-dev:192:42) at Drupal.tableDrag.dropRow (tabledrag.js?v=10.1.0-dev:861:12) at HTMLDocument.<anonymous> (tabledrag.js?v=10.1.0-dev:257:57) at HTMLDocument.dispatch (jquery.min.js?v=3.6.3:2:43336) at y.handle (jquery.min.js?v=3.6.3:2:41320) - Continue the test - it succeeds despite the JavaScript error.
Proposed resolution
The reason why this JavaScript error wasn't caught by js_testing_log.js is because the aforementioned script only records errors handled by the Drupal.throwError() function. The purpose of that function is:
/**
* Helper to rethrow errors asynchronously.
*
* This way Errors bubbles up outside of the original callstack, making it
* easier to debug errors in the browser.
*
* @param {Error|string} error
* The error to be thrown.
*/
Drupal.throwError = function (error) {
setTimeout(() => {
throw error;
}, 0);
};
Also Drupal.throwError() function is only called from within Drupal.attachBehaviors() and Drupal.detachBehaviors() in all of core production JavaScript.
JavaScript errors may occur outside of attach/detach behavior callbacks (e.g., in event handlers, XHR callbacks or in timers). Thus, in order to catch all JavaScript errors it is necessary to setup an error event handler on the window object.
Remaining tasks
User interface changes
API changes
Data model changes
Release notes snippet
FunctionalJavaScript tests will now fail on all JavaScript errors. Previously they only failed on errors thrown by Drupal.throwError()
| Comment | File | Size | Author |
|---|---|---|---|
| #39 | interdiff_30-39.txt | 2.27 KB | murilohp |
| #39 | 3331835-39.patch | 12.04 KB | murilohp |
| #8 | 3331835-some-javascript-errors-8-TEST_ONLY.diff | 3.77 KB | znerol |
Comments
Comment #2
znerol commentedComment #3
znerol commentedComment #4
znerol commentedComment #5
znerol commentedComment #6
znerol commentedComment #7
znerol commentedComment #8
znerol commentedComment #11
spokjeLooks like this fix uncovered a few already failing JS tests.
Comment #12
spokjeThis should leave one failing JS test I believe.
Weird interdiff probably due to #8 being a diff.Comment #13
spokjeThis should leave one failing JS test I hope.
Comment #14
spokjeComment #15
spokjeWell...let's say I replaced some errors with some other errors...
Comment #16
spokjeComment #17
spokjeComment #18
spokjeComment #19
spokjeI'm afraid the last remaining JS test failure needs a bigger JS Brain than mine.
No clue what's going on there.
Comment #20
spokjeGot it, should be no test failures now.
Tentatively raising to Major, since there are quite a few problems with CKEditor5 JS, including a missing JS-dependency, which could potentially has nasty effects.
Comment #21
znerol commentedIt's a bit weird to set that inside the loop. The
failOnJavascriptConsoleErrorsproperty has an effect for the whole test case because it is evaluated in a post-condition (i.e., at the end of the test case).A more defensive approach would be to call
$this->failOnJavaScriptErrors();before theforeachloop in order to catch all potential JavaScript errors from the upper part of the test method. After that set$this->failOnJavascriptConsoleErrors = FALSE;. E.g.:Comment #22
xjmComment #23
spokjeThanks @znerol for the valid remark.
I don't think we need the
$this->failOnJavaScriptErrors();, since tests will fail by default on those (at least after your excellent sleuthing in the related issue as well as in this one).Also added some comments that might help people with a NotSoBigJavaScriptBrain (like me) understand what is being tested.
At the very least I needed some time to grok what was going on.
Comment #24
spokjeRight...
Let's go full mental and a patch as well to the same interdiff twice uploaded...
_facepalms_
Comment #25
spokjeWhen we get to the committing phase we need the whole
*.es6.js-jig, besides not backporting the JS-fail tests on9.x, since #3293090: Fail JavaScript tests on JavaScript errors never landed over there.Comment #26
spokjeMeh, after some more local testing it turns out we _do_ need
$this->failOnJavaScriptErrors();, at least sort of.Also, it looks like changing failOnJavaScriptErrors halfway during a test doesn't work, so I've split up the current test in two tests.
Comment #27
spokjeFixing typo.
Comment #28
smustgrave commentedTested this out by applying this the patch in #27 and the test only patch from [#3305706#comment-14859013] and confirmed I got the error that was expected in [#3305706#comment-14858943]
Comment #29
bnjmnmThis is great! My feedback should be fairly simple to take care of, so I imagine this can be back to RTBC soon. Feel free to ping when that's the case.
👍 I spotted this problem earlier today in an unrelated issue. I support fixing it here since it is in a context where test coverage is introduced that backs up the need to do this.
Null is falsy in JavaScript so this can be
if(form)and will potentially catch other invalid use cases.Similarly, this can just be
if (activeTabLink)You know what I'm gonna say 🙂
Nit: lets leave the line as-is since it's unrelated to the issue scope.
It's kinda evident what this is doing, but lets add the usual docblock with a description
This can all fit on a single line.
Comment #30
smustgrave commentedOnly one I wasn't sure about was #29.1
Comment #32
smustgrave commentedSo changing if (that.autoTags !== null) {
caused a test fail? That doesn't seem right?
Comment #33
smustgrave commentedActually is that the random ckeditor5 error that pops up everywhere?
Looked at maybe 100 review tickets today think I can't read anymore.
Comment #34
spokjeOrdered a re-test and it was indeed a random fail.
Comment #35
smustgrave commentedBack to NR thanks for double checking!
Comment #36
murilohp commentedWhat do you think about using an early return here? IMHO this is better for the complexity, something like:
Just a nitpick here, it seems a comma is missing here, IMO this should be:
"Remove error.log, so we have a clean slate for the next request."
I'll leave as NR since it's just some nitpicks the code looks goods and if you think that we should address the nitpicks I've mentioned here, feel free to update the patch and I can review it again.
Comment #37
bircherI randomly found this issue mentioned on slack.
Unfortunately only after I had spent some time on learning from scratch how to do what this issue is addressing.
But I am very happy to have found this, because apart from the name of the key and the conditional the code I am using to test this on a client project with DTT looks the same.
The nit picks from #36 above are valid, however, to me 1) may be true, but we have the same construct with if(something){ something.stuff;} also in other places of the patch and it makes sense to me also the way the patch describes it. I have no preference of one or the other here. So I set it to RTBC and let the committer decide. The same goes for 2) here a committer may just add the comma. I know good language is something we should strive for, but I would not have spotted the missing comma and I was about to RTBC this issue from my phone on the weekend.
Comment #38
bnjmnmAs much as I hate to switch this back, #36.1 and #36.2 are good suggestions that should probably happen.
It would also be fine to create followup issues for those small changes and switch this back to RTBC.
Comment #39
murilohp commentedSorry, that wasn't my intention :( , I think I can help by just addressing #36 here, I'm not going to move this back to RTBC because I'm addressing my own review here.
Comment #40
bircherI was fine with it before, I am ok with it now too :D
Comment #42
bnjmnm@murilohp zero objection to anything you did here 🙂. It was good feedback and those kind of things absolutely help the maintainability of the code.
@bircher thanks for being at the ready to RTBC again, this really helps momentum
I committed this to 10.1.x. I'm not backporting as this could result in suprise test failures for modules/themes with previously uncaught JS errors. While it's good these errors are now being caught, having them appear in a patch release could be risky + disruptive.