Add tools and scripts for writing and running javascript unit tests

Should we mark #2702747: Add javascript unit testing as a duplicate?

Have added couple packages to work with $, ajax, drupal anonymous functions, etc.
Also another example of test.

I am not sure how to properly use jsdom-global for now, may be we could erase most of the code in boostrap.js file :)

Also what do you think about splitting js files on API files and actual files with behaviors?

Drupal object itself, Drupal.ProgressBar, Drupal.debounce, Drupal.Ajax*, etc. We could something like npm package with API js files.

One part we should take into account in this issue is whether / how to deal with javascript functional testing. At least for now a lot of javascript functional test coverage is missing, and I'm wondering whether moving that to JS would help. In case we decide that we should take into account and probably use a testing framework which can handle both.

There are probably many systems out there, of which one is Nightwatch.

This raises a couple of questions:

• Do we want the functional JS tests in JS or in PHP?
• Would be attract more JS people by not force them to write PHP?
• Would it be a disadvantage to have two systems to write tests in when you want to contribute to Drupal on both PHP and JS?

#13

One part we should take into account in this issue is whether / how to deal with javascript functional testing.

Yes.

At least for now a lot of javascript functional test coverage is missing, and I'm wondering whether moving that to JS would help.

It would possibly make it easier for people who know minor bits about PHP to write functional tests for JavaScript components.

Do we want the functional JS tests in JS or in PHP?

In JavaScript.

Would be attract more JS people by not force them to write PHP?

Yes.

Would it be a disadvantage to have two systems to write tests in when you want to contribute to Drupal on both PHP and JS?

Possibly?, but probably only briefly.

I've been thinking a bit about this and I'll just share my view on this. Do note that I'm relatively new to Drupal and testing such a system but have more experience in projects for work with a QA specialist. I have been quite active in the PHPUnit initiative in the last couple of months so I do feel that it's useful to share my view :)

So what are we actually testing?
We say and assume we're testing JavaScript with the JavaScriptTestBase tests, I'm not 100% sure on that though. Why? We're still doing functional tests, but this time in a real browser instead of a simulated one. I think that we should therefore look at this differently, don't focus on JavaScript but on implementing real-world tests, that's what we're trying to do.

The reason why we need it is still the same, a simulated browser is not enough for testing rich user interfaces that utilize browser functions like JavaScript or CSS.

The question then is how to test it
Currently we’re running tests in PHP, which to me is fine because the other functional testing (BrowserTestBase) is PHP as well. This way we have a similar environment for both types of test and it’s easy to move a BrowserTestBase to JavascriptTestBase.

To me functional testing in JavaScript doesn’t feel right (yet?), to me we'll only move the responsibility to an area where we have even less people interested in functional testing. I'm not convinced either that it's better to functional test in JavaScript and that deep JavaScript knowledge is required to do it.

To clarify and prevent any misunderstandings: I do agree on needing unit tests in Javascript and I like the direction for this in this issue :)

Another option would be to use Selenium for these kinds of test (I think this was discussed in #825436: Create selenium-RC PIFR plugin for full functional testing but rejected at the time). I'm not very familiar with Selenium but it is a much used tool for functional testing by the QA specialist. It would actually open up a whole new way of contributing to Drupal for QA specialist. Writing PHP is difficult for them but Selenium tests are well within their comfort zone. It would still cause a split in how functional tests are written though with BrowserTests.

I opened up an issue about leveraging nightwatch for functional test coverage on #2869825: Leverage JS for JS testing (using nightwatch)

What blocking this move forward?

I thought #13, #14, #15, #16 are unrelated to JS unit tests.

@droplet
Well, at least nightwatch would add a unit test runner as well.

I'm thinking if we could make a shortcut as a temp workaround.

Most of these engines (Jest / Mocha / Nightwatch) are shared a similar syntax. There's codemods to migrate them ( https://www.npmjs.com/package/jest-codemods ). Usually, with my own or others experiences on the net, migration is pretty easy and fast.

We actually could start to use Mocha and load a HTML page in the current test env:
https://mochajs.org/#running-mocha-in-the-browser

A pure JS unit test really help for these issues: (More productive also)
#2880168: Refactor drupal.js to take advantage of new language features.
#736066: ajax.js insert command sometimes wraps content in a div, potentially producing invalid HTML and other bugs
#1988968: Drupal.ajax does not guarantee that "add new JS file to page" commands have finished before calling said JS

@dawehner,

Do you mean a parallel runner for cross-platform?

So, we can now do this, maybe not yet with all sorts of fancy stuff, but the basics seem to work fine. Would it be an idea to add an example for this to core? So something like this.

+++ b/core/tests/Drupal/Nightwatch/Scripts/unitTestScript.js
index 0000000000..6dff4a4f13
--- /dev/null

--- /dev/null
+++ b/core/tests/Drupal/Nightwatch/Tests/testUnitTest.js

+++ b/core/tests/Drupal/Nightwatch/Tests/testUnitTest.js
+++ b/core/tests/Drupal/Nightwatch/Tests/testUnitTest.js
@@ -0,0 +1,21 @@

@@ -0,0 +1,21 @@
+const assert = require('assert');
+const testScript = require('../Scripts/unitTestScript');
+

The current tests in nightwatch are in the Tests folder. Given that I'm wondering whether moving unit tests to its own folder would be sensible.

Yeah, let's move it, moved it to a subfolder, might be an idea to move the others to a 'functional' folder. Also renamed the test to be more in line with the existing ExampleTest.

We probably want to document unit tests at least in https://www.drupal.org/docs/8/testing/javascript-testing-using-nightwatch. Do you think we need a change record?

Updated the docs with the example.

I don't think we want to do a release note for this, since we are not actually adding any new functionality here, but happy to be overruled on that :)

I think we should move forward with this

Marking as RTBC, but it likely will need a re-roll, but let's see what the bot says

I'm kind of biased, but I like the ergonomics of the testing-library suite, and specifically twig-testing-library, because we can load twig for the markup, but yeah, that's a whole load of extra dependencies.

Updated patch will resolve CS issues

I'm not sure if it's something we should do here or not but we already tried to use Nightwatch for unit testing CKEditor 5 code. However, that turned out quite difficult because of the lack of support for ESM: #3247647: Document how to run Nightwatch unit tests with code using ESM. I'm wondering if we should figure out how to do that before we commit to using Nightwatch for this.

Would be also great if we could update the issue summary with the proposed solution.

For IS update requested in #36