The 2026 Drupal Business Survey is open. Your response will help shape the future of the Drupal business ecosystem.Problem/Motivation
Right now, if a single test case in a Nightwatch test fails, all other test cases in that test are skipped.
This is unlike every other kind of test in Drupal core: unit, kernel, functional and functional JS tests always run all tests.
This is not just inconsistent, it also lengthens the feedback loop.
Steps to reproduce
Any Nightwatch test case that fails that isn't the last case in that test. Either on d.o's test infrastructure or locally by using yarn test:nightwatch --env local.
Proposed resolution
Add skip_testcases_on_fail: false to core/tests/Drupal/Nightwatch/nightwatch.conf.js.
See https://nightwatchjs.org/gettingstarted/configuration/.
Remaining tasks
None.
User interface changes
None.
API changes
None.
Data model changes
None.
Release notes snippet
None.
| Comment | File | Size | Author |
|---|---|---|---|
| #4 | 3217355-4.patch | 1.31 KB | wim leers |
| #2 | 3217355-2.patch | 785 bytes | wim leers |
Comments
Comment #2
wim leersComment #3
wim leersComment #4
wim leers#2 failed on a silly coding style thing.
However, I noticed that one extra thing must be done for Drupal nightwatch tests that if they are reinstalling Drupal in each test case: cookies must be deleted. Otherwise the user will be treated as still logged in (since the session cookie exists) but session cookies do not actually apply to this newly installed Drupal site.
Comment #5
MixologicI think this is a good change. We dont really save any testbot minutes by failing fast in nightwatch testing because its at the very end of the suite, so it likely causes us to use *more* minutes than we otherwise would with this setting.
(we used to have so few nightwatch tests as to not really matter one way or the other, so its great that we need to consider this now)
Comment #6
longwaveMakes sense to be consistent with PHPUnit, and as #5 says these run at the very end of the process so it's definitely better that if there are multiple separate Nightwatch fails that we get to see them all in one go instead of fixing one and then discovering the next.
Comment #8
wim leersFailed because MySQL server went on vacation.
Comment #10
catchCommitted 94acb91 and pushed to 9.3.x. Thanks!