Problem/Motivation

Now that #2469713: Step 2: Create a JavaScriptTestBase using PhantomJs Driver/Binary is in, we have some way to do end to end testing of our functionality, but if we want to improve our javascript, we better write good tests.

Proposed resolution

The area of javascript testing is scattered to say the least, here is the combination I went with:

  • Mocha: Its a testing framework which allows you to define and run tests. Its based upon nodejs by default, but is pretty flexible
  • Chai: A assertion library, which is pretty flexible as it supports should, expects and assert notation
  • sinon: A mocking framework

Questions

  • Do we want unit tests? Everyone agrees with that
  • Do we want to add the dependencies into core? Noone wants that, but let's add a shrinkwrap file (lock file): https://docs.npmjs.com/cli/shrinkwrap
  • Is CLI based unit testing okay, or do we want more, like browserbased testing with karma? Maybe on the longrun, but let's get something running now and not later
  • Is it okay to start without testbot integration? Yes, its okay to make progress, obviously on the longrun we want support on the testbot.

How to use the unit tests

The patch ships with a readme:

### Javascript
+
+First you need to install the dependencies using npm, see (Installing NPM)$https://docs.npmjs.com/getting-started/installing-node].
+
+```
+cd core/tests/js
+npm install
+```
+
+Once you have installed it, you can run the tests using
+
+```
+node_modules/mocha/bin/mocha
+```

Remaining tasks

User interface changes

API changes

Data model changes

Members fund testing for the Drupal project. Drupal Association Learn more

Comments

dawehner created an issue. See original summary.

dawehner’s picture

Issue summary: View changes
FileSize
7.04 MB
dawehner’s picture

larowlan’s picture

+1 for mocha from me

But there are infrastructure issues with needing node on our bots and collecting results

nod_’s picture

Looking back at #2232271: [Meta] Use Behat for validation testing, functional testing would be more valuable today as a maintainer. Our JS is not unit-testable as it is written. Apart from some trivial functions in drupal.js there is not much we could unit-test: http://read.theodoreb.net/drupal-jsapi/Drupal.html And it's all tied up to the dom anyway.

dawehner’s picture

Status: Active » Needs review
FileSize
2.62 KB

Looking back at #2232271: [Meta] Use Behat for validation testing, functional testing would be more valuable today as a maintainer.

We have that now with #2469713: Step 2: Create a JavaScriptTestBase using PhantomJs Driver/Binary and yeah in think totally that for end-to-end testing, its good that we focus on that.

Our JS is not unit-testable as it is written. Apart from some trivial functions in drupal.js there is not much we could unit-test: http://read.theodoreb.net/drupal-jsapi/Drupal.html And it's all tied up to the dom anyway.

There is this interesting strong correlation between good architected and especially well maintainable code and unit testable code. So having a unit testing system for JS can help us to provide reusable piece of code. This totally was visible in our PHP code.

dawehner’s picture

Issue summary: View changes
dawehner’s picture

Issue tags: +DrupalBCDays
dawehner’s picture

Issue tags: +JavaScript
nod_’s picture

  1. Do we want unit tests?
    yes
  2. Do we want to add the dependencies?
    no, and we should gitignore core/tests/js/node_modules. The changes to npm modules are very relevant for this: http://blog.npmjs.org/post/141905368000/changes-to-npms-unpublish-policy, as long as we don't mess with obscure npm modules we should be safe.
  3. Is CLI based unit testing okay, or do we want more, like browserbased testing with karma
    We can start off with CLI, the hard part will be refactoring our code and writing tests, should be enough code out there to deal with infrastructure
dawehner’s picture

Issue summary: View changes

Talked with @nod_ and @klausi, so here is a bit of an issue summary update

dawehner’s picture

Added a shrinkwrap file and added node_modules to the gitignore.

droplet’s picture

should use `npm script` (https://docs.npmjs.com/cli/run-script) instead of looong ReadMe.txt

npm install
npm run test

also adding `phantomjs` to package.json. (https://github.com/Medium/phantomjs)

Thinking in future, I think we should place package.json in root dir.

Then we can add:

npm run eslint
npm run styleline
..
etc

alexpott’s picture

Is it okay to start without testbot integration? Yes, its okay to make progress, obviously on the longrun we want support on the testbot.

I've got to disagree. I think we need to have this otherwise everyone has to run locally when making a javascript change, and the committer too. This is really painful atm with coding standards we are always regressing.

nod_’s picture

dawehner’s picture

I've got to disagree. I think we need to have this otherwise everyone has to run locally when making a javascript change, and the committer too. This is really painful atm with coding standards we are always regressing.

I mean no question, on the longrun there is no question about it. To be fair though, we have much more code, than tests at the moment.

Here is some continued research which adds some helper script that allows to run tests from modules as well. This doesn't yet integrate into run-tests.sh yet, but I guess a quick workaround could be to iterate over each module and try to run their javascript tests.

dawehner’s picture

Just some continues work to add some kind of scanning for available JS tests ... even I'm really not convinced that we should actually do it. For me the integration with the testbot for now could be just a script that runs all of the tests, and collects the JUNIT xml files, and well, be happy with it. Adding full support for running individual tests from within run-tests.sh is IMHO quite pointless and just a waste of time.

Status: Needs review » Needs work

The last submitted patch, 17: 2702747-17.patch, failed testing.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.0-beta1 was released on August 3, 2016, which means new developments and disruptive changes should now be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

dawehner’s picture

Note: To be honest, this entire integration with PHPUnit and JS was mostly to get the simpletest UI working with the JS tests. I honestly don't believe in the usefullness of it, but there are a many people who care a lot about it: #2750461: Remove Simpletest UI because we don't want to maintain a graphical test runner

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

dawehner’s picture

Status: Needs work » Closed (duplicate)