Problem/Motivation
The output of the nightwatch job on GitLab CI is not easily readable.
After the first test, the output seems to lack newlines:
┌ ────────────────── ✔ default: Tests/toolbarApiTest.js ─────────────────────┐│ ││ ││ [Tests/Toolbar Api Test] Test Suite ││ ────────────────────────────────────────────────────────────────────── ││ Using: chrome (106.0.5249.103) on LINUX. ││ - Loading url: http://localhost/subdirectory ││ ℹ Loaded url http://localhost/subdirectory in 3603ms ││ - Loading url: http://localhost/subdirectory/user/reset/1/1699769393/rAx ││ XXMRqoilkKkm9F1ftirF9ByGekM_5ilpoeJ7cOds/login ││ ℹ Loaded url http://localhost/subdirectory/user/reset/1/1699769393/rAxXX ││ MRqoilkKkm9F1ftirF9ByGekM_5ilpoeJ7cOds/login ││ in 1284ms ││ - Loading url: http://localhost/subdirectory/admin/modules ││ ℹ Loaded url http://localhost/subdirectory/admin/modules in 1468ms ││ ✔ Element <form.system-modules [name="modules[toolbar][enable]"]> was ││ visible after 25 milliseconds. ││ ✔ Element <#system-modules-confirm-form> was present after 1542 ││ milliseconds. ││ ✔ Element <form.system-modules ││ [name="modules[toolbar][enable]"]:disabled> was present after 45 ││ milliseconds. ││ - Loading url: http://localhost/subdirectory/user/logout ││ ℹ Loaded url http://localhost/subdirectory/user/logout in 245ms ││ - Loading url: http://localhost/subdirectory/user/reset/1/1699769407/C9F ││ cHTHML3hD-dyWjgTiXgMSfGlriQJUzj-FSH0GTeE/login ││ ℹ Loaded url http://localhost/subdirectory/user/reset/1/1699769407/C9FcH ││ THML3hD-dyWjgTiXgMSfGlriQJUzj-FSH0GTeE/login ││ in 356ms ││ - Loading url: http://localhost/subdirectory/admin/people/roles/add ││ ℹ Loaded url http://localhost/subdirectory/admin/people/roles/add in ││ 421ms ││ ✔ Expected element <.user-role-form .machine-name-value> to be visible ││ in 2000ms (67ms) ││ - Loading url: http://localhost/subdirectory/admin/people/permissions ││ ℹ Loaded url http://localhost/subdirectory/admin/people/permissions in ││ 1883ms ││ - Loading url: http://localhost/subdirectory/admin/people/permissions ││ ℹ Loaded url http://localhost/subdirectory/admin/people/permissions in ││ 731ms ││ - Loading url: http://localhost/subdirectory/user/logout ││ ℹ Loaded url http://localhost/subdirectory/user/logout in 164ms ││ - Loading url: http://localhost/subdirectory/user/reset/1/1699769414/I2A ││ V9n1kyOJ9KG70PnB6GJG7qr9sFcw9Ei6gT-6rqNI/login ││ ℹ Loaded url http://localhost/subdirectory/user/reset/1/1699769414/I2AV9 ││ n1kyOJ9KG70PnB6GJG7qr9sFcw9Ei6gT-6rqNI/login ││ in 269ms ││ - Loading url: http://localhost/subdirectory/admin/people/create ││ ℹ Loaded url http://localhost/subdirectory/admin/people/create in 389ms ││ ✔ User "user" was created successfully (723ms) ││ (node:117) MaxListenersExceededWarning: Possible EventEmitter memory ││ leak detected. 11 error listeners added to [CommandLoader]. Use ││ emitter.setMaxListeners() to increase limit ││ (Use `node --trace-warnings ...` to show where the warning was created) ││ - Loading url: http://localhost/subdirectory/user/logout ││ ℹ Loaded url http://localhost/subdirectory/user/logout in 145ms ││ - Loading url: http://localhost/subdirectory/user/login ││ ℹ Loaded url http://localhost/subdirectory/user/login in 102ms ││ (node:117) MaxListenersExceededWarning: Possible EventEmitter memory ││ leak detected. 11 error listeners added to [CommandLoader]. Use ││ emitter.setMaxListeners() to increase limit ││ ✔ Passed [equal]: The user "user" was logged in. ││ ││ – Drupal.Toolbar.models ││ - Loading url: http://localhost/subdirectory/ ││ ℹ Loaded url http://localhost/subdirectory/ in 267ms ││ ✔ Element <#toolbar-administration> was present after 18 milliseconds. ││ ✔ Passed [deepEqual]: Keys to check match ││ ✔ Passed [equal]: has menu model ││ ✔ Passed [equal]: menu model is an object ││ ✔ Passed [equal]: has toolbar model ││ ✔ Passed [equal]: toolbar model is an object ││ ✔ Passed [equal]: get("activeTab") has expected result ││ ✔ Passed [equal]: get("activeTray") has expected result ││ ✔ Passed [equal]: get("isOriented") has expected result ││ ✔ Passed [equal]: get("isFixed") has expected result ││ ✔ Passed [equal]: get("areSubtreesLoaded") has expected result ││ ✔ Passed [equal]: get("isViewportOverflowConstrained") has expected ││ result ││ ✔ Passed [equal]: get("orientation") has expected result ││ ✔ Passed [equal]: get("locked") has expected result ││ ✔ Passed [equal]: get("isTrayToggleVisible") has expected result ││ ✔ Passed [equal]: get("height") has expected result ││ ✔ Passed [equal]: get("offsets") bottom has expected result ││ ✔ Passed [equal]: get("offsets") left has expected result ││ ✔ Passed [equal]: get("offsets") right has expected result ││ ✔ Passed [equal]: get("offsets") top has expected result ││ ✔ Passed [equal]: get("subtrees") has expected result ││ ✔ default: Tests/toolbarApiTest.js [Tests/Toolbar Api Test] ││ Drupal.Toolbar.models (471ms) ││ ││ – Change tab ││ - Loading url: http://localhost/subdirectory/ ││ ℹ Loaded url http://localhost/subdirectory/ in 185ms ││ ✔ Element <#toolbar-administration> was present after 14 milliseconds. ││ ✔ Passed [deepEqual]: Keys to check match ││ ✔ Passed [equal]: has menu model ││ ✔ Passed [equal]: menu model is an object ││ ✔ Passed [equal]: has toolbar model ││ ✔ Passed [equal]: toolbar model is an object ││ ✔ Passed [equal]: get("activeTab") has expected result ││ ✔ Passed [equal]: get("activeTray") has expected result ││ ✔ default: Tests/toolbarApiTest.js [Tests/Toolbar Api Test] Change tab ││ (577ms) ││ ││ – Change orientation ││ - Loading url: http://localhost/subdirectory/ ││ ℹ Loaded url http://localhost/subdirectory/ in 227ms ││ ✔ Element <#toolbar-administration> was present after 42 milliseconds. ││ ✔ Passed [deepEqual]: Keys to check match ││ ✔ Passed [equal]: get("orientation") has expected result ││ ✔ Passed [equal]: changing orientation has expected result ││ ✔ default: Tests/toolbarApiTest.js [Tests/Toolbar Api Test] Change ││ orientation (587ms) ││ ││ – Open submenu ││ - Loading url: http://localhost/subdirectory/ ││ ℹ Loaded url http://localhost/subdirectory/ in 243ms ││ ✔ Element <#toolbar-administration> was present after 21 milliseconds. ││ ✔ Passed [deepEqual]: Keys to check match ││ ✔ Passed [equal]: get("orientation") has expected result ││ ✔ Passed [equal]: opening submenu has expected result ││ ✔ default: Tests/toolbarApiTest.js [Tests/Toolbar Api Test] Open submenu ││ (695ms) ││ │└──────────────────────────────────────────────────────────────────────────────┘
https://git.drupalcode.org/project/drupal/-/jobs/316331#L149 (Yes, this is all just one output line...)
Screenshot here: https://www.drupal.org/files/issues/2023-11-12/before.jpg
Problem comes out of a dependency of nightwatch called boxen (https://www.npmjs.com/package/boxen).
This was fixed in version 7.0.1, however nightwatch is (and also will be in 3.x) using 5.1.2.
Fix PR is here: https://github.com/sindresorhus/boxen/pull/81
Steps to reproduce
Proposed resolution
viewing the source of boxen implies we might be able to use the COLUMNS environment variable to trick it into printing newlines.
@longwave in #14
Also turn off the very verbose output to keep all the tests visible in the GitLab output window-thingy
Remaining tasks
User interface changes
API changes
Data model changes
Release notes snippet
| Comment | File | Size | Author |
|---|---|---|---|
| #3 | before.jpg | 628.01 KB | spokje |
Issue fork drupal-3401047
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #2
spokjeComment #3
spokjeComment #4
spokjeComment #5
spokjeComment #7
spokjeComment #8
smustgrave commentedUse patch-packages all the time! Think this will need submaintainer or framework approval for the new packages.
Comment #9
needs-review-queue-bot commentedThe Needs Review Queue Bot tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".
This does not mean that the patch needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.
Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.
Comment #10
spokjeRebased
Comment #11
spokjeComment #14
longwaveTrying an alternative approach, viewing the source of
boxenimplies we might be able to use theCOLUMNSenvironment variable to trick it into printing newlines.Comment #15
needs-review-queue-bot commentedThe Needs Review Queue Bot tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".
This does not mean that the patch needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.
Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.
Comment #17
kostyashupenkoRebased, now patch from 5534 mr can be applied
Comment #18
smustgrave commentedDon't think needs submaintainer if not adding new package.
Checked MR5534 and output isn't 2 columns anymore but the tests don't make sense. Just a bunch of "true ok"
Comment #19
longwaveThat is because the tabbable test doesn't provide helpful assertion messages in two of the three assertions in the loop:
Comment #20
smustgrave commentedAh okay, know we don't write a lot of nightwatch tests but may be worth opening a follow up to provide better mesages.
Comment #21
quietone commentedThere are 2 MRs here, which one has been set to RTBC? I think it is the later one, which means the Issue Summary is out of date. Tagging for that as well.
Comment #24
spokje*grmbl* Bloody newbies, always doing a drive-by MR *grmbl*
Anyway....
- Closed my MR
- Rebased longwave's MR
- Removed the unneeded/not working setting of COLUMNS as environment variable
- Updated IS
Since there's technically a code change since the last RTBC (the removed line), putting this back to NR
Comment #25
spokjeHmmm, only just realized that we're now not able to see the approximately first third of the tests...
We can only scroll upto around
oliveroStickyHeaderToggleTestWe seem to run out of the max amount of lines allowed on GitLab?
Is there a way to up that number for nightwatch log only?
Personally, ATM I think I rather have (faux) two columns with all the results than correct aligned 2/3 of the testlog
Comment #26
longwaveIs it really helpful to show every passed assertion anyway? Should we set detailed_output to false? https://nightwatchjs.org/guide/configuration/customising-test-output.html
Or we could set disable_colors to true and then the raw log should be more readable?
Comment #27
spokjeAh...
I was somehow under the impression that somebody somewhere explicitly wanted this very verbose output.
But but, I like the pretty colors....
Let's kill
detailed_outputand that should be more than enough to keep all the tests in view (and color ;)Comment #28
spokjeComment #29
smustgrave commentedAdding variable seems correct to me.
Comment #30
spokjeWhilst we still have the very verbose output, let's use that to check if all deprecations are replaced in #3413135: Replace deprecated functions in Nightwatch tests.
Postponing on that issue.
Comment #31
spokje#3413135: Replace deprecated functions in Nightwatch tests landed.
As a result, the lack of deprecation notices in the log now makes it short enough to be fully viewable in GitLab again.
Since I don't see a massive increase in Nightwatch tests/added assertions to existing test, I removed the
detailed_output: false.Back to NR.
Comment #32
fjgarlin commentedThe change seems simple enough and the output is defo better (see https://git.drupalcode.org/issue/drupal-3401047/-/jobs/633328).
I don't know how much more we can control it, so I'm setting this RTBC.
Comment #33
andypostBlocker reverted #3413135: Replace deprecated functions in Nightwatch testsSorry mixed with #3413665: Enable modules through Nightwatch API when not testing module enabling
Comment #37
lauriiiCommitted 869929d and pushed to 11.x. Also cherry-picked to 10.2.x. Thanks!