Problem/Motivation

Since #3091870: Fail Functional Javascript tests that throw Javascript errors all JavaScript errors thrown during test execution are supposed to fail the test run. However, the JavaScript error produced by patch from #3305706-30: Uncaught TypeError: this.$pluginSelect.find(...)[0] is undefined. Does not fail the test.

Steps to reproduce

  • Apply patch from #3305706-30: Uncaught TypeError: this.$pluginSelect.find(...)[0] is undefined.
  • Ensure that test run is paused after the dragTo() call in EntityDisplayTest::testHiddenExtraFields() (using a breakpoint or a strategically placed wait as recommended in the docs.)
  • Run the test:
    ./vendor/bin/phpunit -v --filter testHiddenExtraFields core/modules/field_ui/tests/src/FunctionalJavascript/EntityDisplayTest.php
  • Inspect the browser console. It will should show the following error:
    field_ui.js?v=10.1.0-dev:371 Uncaught TypeError: Cannot read properties of undefined (reading 'value')
        at Drupal.fieldUIDisplayOverview.field.regionChange (field_ui.js?v=10.1.0-dev:371:51)
        at Drupal.tableDrag.onDrop (field_ui.js?v=10.1.0-dev:192:42)
        at Drupal.tableDrag.dropRow (tabledrag.js?v=10.1.0-dev:861:12)
        at HTMLDocument.<anonymous> (tabledrag.js?v=10.1.0-dev:257:57)
        at HTMLDocument.dispatch (jquery.min.js?v=3.6.3:2:43336)
        at y.handle (jquery.min.js?v=3.6.3:2:41320)
    
  • Continue the test - it succeeds despite the JavaScript error.

Proposed resolution

The reason why this JavaScript error wasn't caught by js_testing_log.js is because the aforementioned script only records errors handled by the Drupal.throwError() function. The purpose of that function is:

  /**
   * Helper to rethrow errors asynchronously.
   *
   * This way Errors bubbles up outside of the original callstack, making it
   * easier to debug errors in the browser.
   *
   * @param {Error|string} error
   *   The error to be thrown.
   */
  Drupal.throwError = function (error) {
    setTimeout(() => {
      throw error;
    }, 0);
  };

Also Drupal.throwError() function is only called from within Drupal.attachBehaviors() and Drupal.detachBehaviors() in all of core production JavaScript.

JavaScript errors may occur outside of attach/detach behavior callbacks (e.g., in event handlers, XHR callbacks or in timers). Thus, in order to catch all JavaScript errors it is necessary to setup an error event handler on the window object.

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

FunctionalJavaScript tests will now fail on all JavaScript errors. Previously they only failed on errors thrown by Drupal.throwError()

Comments

znerol created an issue. See original summary.

znerol’s picture

Issue summary: View changes
znerol’s picture

Issue summary: View changes
znerol’s picture

Issue summary: View changes
znerol’s picture

Status: Active » Needs review
StatusFileSize
new5.77 KB
new3.76 KB
znerol’s picture

znerol’s picture

znerol’s picture

The last submitted patch, 7: 3331835-some-javascript-errors-7-TEST_ONLY.diff, failed testing. View results

Status: Needs review » Needs work

The last submitted patch, 8: 3331835-some-javascript-errors-8-TEST_ONLY.diff, failed testing. View results

spokje’s picture

Looks like this fix uncovered a few already failing JS tests.

spokje’s picture

StatusFileSize
new4.27 KB
new0 bytes

This should leave one failing JS test I believe.

Weird interdiff probably due to #8 being a diff.

spokje’s picture

StatusFileSize
new8.3 KB
new2.09 KB

This should leave one failing JS test I hope.

spokje’s picture

StatusFileSize
new7.6 KB
new1.54 KB
spokje’s picture

Well...let's say I replaced some errors with some other errors...

spokje’s picture

StatusFileSize
new312 bytes
new8.02 KB
spokje’s picture

StatusFileSize
new1.65 KB
new9.78 KB
spokje’s picture

StatusFileSize
new503 bytes
new9.78 KB
spokje’s picture

I'm afraid the last remaining JS test failure needs a bigger JS Brain than mine.
No clue what's going on there.

spokje’s picture

Priority: Normal » Major
Status: Needs work » Needs review
StatusFileSize
new10.72 KB
new796 bytes

Got it, should be no test failures now.

Tentatively raising to Major, since there are quite a few problems with CKEditor5 JS, including a missing JS-dependency, which could potentially has nasty effects.

znerol’s picture

       // The exceptions are expected. Do not interpret them as a test failure.
+      $this->failOnJavascriptConsoleErrors = FALSE;
       // Not using File API; a potential error must trigger a PHP warning.

It's a bit weird to set that inside the loop. The failOnJavascriptConsoleErrors property has an effect for the whole test case because it is evaluated in a post-condition (i.e., at the end of the test case).

A more defensive approach would be to call $this->failOnJavaScriptErrors(); before the foreach loop in order to catch all potential JavaScript errors from the upper part of the test method. After that set $this->failOnJavascriptConsoleErrors = FALSE;. E.g.:

     $div1 = $this->assertSession()->waitForElement('css', "div#ajax_checkbox_value:contains('unchecked')");
     $this->assertNotNull($div1, 'DataCommand updates the DOM as expected when a checkbox is de-selected');
 
+    // Assert on any JavaScript errors potentially thrown until here. After that
+    // disable error capture for the rest of the test method.
+    $this->failOnJavaScriptErrors();
+    $this->failOnJavascriptConsoleErrors = FALSE;
 
     // Verify that AJAX elements with invalid callbacks return error code 500.
     // Ensure the test error log is empty before these tests.
     $this->assertFileDoesNotExist(DRUPAL_ROOT . '/' . $this->siteDirectory . '/error.log');
xjm’s picture

Priority: Major » Critical
spokje’s picture

StatusFileSize
new1.9 KB
new1.9 KB

Thanks @znerol for the valid remark.

I don't think we need the $this->failOnJavaScriptErrors();, since tests will fail by default on those (at least after your excellent sleuthing in the related issue as well as in this one).

Also added some comments that might help people with a NotSoBigJavaScriptBrain (like me) understand what is being tested.
At the very least I needed some time to grok what was going on.

spokje’s picture

StatusFileSize
new11.76 KB

Right...
Let's go full mental and a patch as well to the same interdiff twice uploaded...
_facepalms_

spokje’s picture

When we get to the committing phase we need the whole *.es6.js-jig, besides not backporting the JS-fail tests on 9.x, since #3293090: Fail JavaScript tests on JavaScript errors never landed over there.

spokje’s picture

StatusFileSize
new1.27 KB
new12.52 KB

Meh, after some more local testing it turns out we _do_ need $this->failOnJavaScriptErrors();, at least sort of.

Also, it looks like changing failOnJavaScriptErrors halfway during a test doesn't work, so I've split up the current test in two tests.

spokje’s picture

StatusFileSize
new620 bytes
new12.52 KB

Fixing typo.

smustgrave’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: +Needs Review Queue Initiative

Tested this out by applying this the patch in #27 and the test only patch from [#3305706#comment-14859013] and confirmed I got the error that was expected in [#3305706#comment-14858943]

bnjmnm’s picture

Status: Reviewed & tested by the community » Needs work

This is great! My feedback should be fairly simple to take care of, so I imagine this can be back to RTBC soon. Feel free to ping when that's the case.

  1. +++ b/core/modules/ckeditor5/ckeditor5.libraries.yml
    @@ -110,6 +110,7 @@ internal.admin:
    +    - core/drupal.announce
    

    👍 I spotted this problem earlier today in an unrelated issue. I support fixing it here since it is in a context where test coverage is introduced that backs up the need to do this.

  2. +++ b/core/modules/ckeditor5/js/ckeditor5.admin.js
    @@ -652,17 +652,19 @@
    +        if (form !== null) {
    

    Null is falsy in JavaScript so this can be if(form) and will potentially catch other invalid use cases.

  3. +++ b/core/modules/ckeditor5/js/ckeditor5.admin.js
    @@ -714,7 +716,9 @@
    +            if (activeTabLink !== null) {
    

    Similarly, this can just be if (activeTabLink)

  4. +++ b/core/modules/filter/filter.filter_html.admin.js
    @@ -114,12 +114,14 @@
    +        if (that.autoTags !== null) {
    

    You know what I'm gonna say 🙂

  5. +++ b/core/tests/Drupal/FunctionalJavascriptTests/Ajax/FormValuesTest.php
    @@ -34,7 +34,6 @@ protected function setUp(): void {
    -
    

    Nit: lets leave the line as-is since it's unrelated to the issue scope.

  6. +++ b/core/tests/Drupal/FunctionalJavascriptTests/JavascriptErrorsTest.php
    @@ -39,6 +39,21 @@ public function testJavascriptErrors(): void {
    +  public function testJavascriptErrorsAsync(): void {
    

    It's kinda evident what this is doing, but lets add the usual docblock with a description

  7. +++ b/core/tests/Drupal/FunctionalJavascriptTests/JavascriptErrorsTest.php
    @@ -39,6 +39,21 @@ public function testJavascriptErrors(): void {
    +    // detected.
    

    This can all fit on a single line.

smustgrave’s picture

Status: Needs work » Needs review
StatusFileSize
new2.72 KB
new12.81 KB

Only one I wasn't sure about was #29.1

Status: Needs review » Needs work

The last submitted patch, 30: 3331835-30.patch, failed testing. View results

smustgrave’s picture

So changing if (that.autoTags !== null) {
caused a test fail? That doesn't seem right?

smustgrave’s picture

Actually is that the random ckeditor5 error that pops up everywhere?

Looked at maybe 100 review tickets today think I can't read anymore.

spokje’s picture

Ordered a re-test and it was indeed a random fail.

smustgrave’s picture

Status: Needs work » Needs review

Back to NR thanks for double checking!

murilohp’s picture

  1. +++ b/core/modules/ckeditor5/js/ckeditor5.admin.js
    @@ -652,17 +652,19 @@
    +        if (form) {
    

    What do you think about using an early return here? IMHO this is better for the complexity, something like:

    if (!form) {
      return;
    }
    // Get the current stored UI state as an object.
    const currentStates = form.hasAttribute('data-drupal-ui-state')
      ? JSON.parse(form.getAttribute('data-drupal-ui-state'))
      : {};
    
    // Store the updated UI state object as an object literal in the parent
    // form's 'data-drupal-ui-state' attribute.
    form.setAttribute(
      'data-drupal-ui-state',
      JSON.stringify({ ...currentStates, ...states }),
    );
    
  2. +++ b/core/tests/Drupal/FunctionalJavascriptTests/Ajax/FormValuesTest.php
    @@ -74,10 +85,11 @@ public function testSimpleAjaxFormValue() {
    +      // Remove error.log so we have a clean slate for the next request.
    

    Just a nitpick here, it seems a comma is missing here, IMO this should be:

    "Remove error.log, so we have a clean slate for the next request."

I'll leave as NR since it's just some nitpicks the code looks goods and if you think that we should address the nitpicks I've mentioned here, feel free to update the patch and I can review it again.

bircher’s picture

Status: Needs review » Reviewed & tested by the community

I randomly found this issue mentioned on slack.
Unfortunately only after I had spent some time on learning from scratch how to do what this issue is addressing.
But I am very happy to have found this, because apart from the name of the key and the conditional the code I am using to test this on a client project with DTT looks the same.
The nit picks from #36 above are valid, however, to me 1) may be true, but we have the same construct with if(something){ something.stuff;} also in other places of the patch and it makes sense to me also the way the patch describes it. I have no preference of one or the other here. So I set it to RTBC and let the committer decide. The same goes for 2) here a committer may just add the comma. I know good language is something we should strive for, but I would not have spotted the missing comma and I was about to RTBC this issue from my phone on the weekend.

bnjmnm’s picture

Status: Reviewed & tested by the community » Needs work

As much as I hate to switch this back, #36.1 and #36.2 are good suggestions that should probably happen.

It would also be fine to create followup issues for those small changes and switch this back to RTBC.

murilohp’s picture

Status: Needs work » Needs review
StatusFileSize
new12.04 KB
new2.27 KB

As much as I hate to switch this back

Sorry, that wasn't my intention :( , I think I can help by just addressing #36 here, I'm not going to move this back to RTBC because I'm addressing my own review here.

bircher’s picture

Status: Needs review » Reviewed & tested by the community

I was fine with it before, I am ok with it now too :D

  • bnjmnm committed 778bdaa6 on 10.1.x
    Issue #3331835 by Spokje, znerol, smustgrave, murilohp, bircher: Some...
bnjmnm’s picture

Issue summary: View changes
Status: Reviewed & tested by the community » Fixed

@murilohp zero objection to anything you did here 🙂. It was good feedback and those kind of things absolutely help the maintainability of the code.

@bircher thanks for being at the ready to RTBC again, this really helps momentum

I committed this to 10.1.x. I'm not backporting as this could result in suprise test failures for modules/themes with previously uncaught JS errors. While it's good these errors are now being caught, having them appear in a patch release could be risky + disruptive.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.