Problem/Motivation

By @finnsky in #3391723: [PLAN] Accessibility review for new Navigation bar:

We could add Nightwatch tests because we have automated Nightwatch a11y tests in core: #3293469: Automated A11y tests in Nightwatch. Added simple tests using it.

How to start with Nightwatch testing can be found in `core/tests/README.md` file from Drupal core.

This issue should be focused on implementing a first simple test, like "(If logo) it takes you to the homepage" and define the structure the rest of the tests will follow.

CommentFileSizeAuthor
#33 3393400-nr-bot.txt90 bytesneeds-review-queue-bot

Issue fork drupal-3393400

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Issue fork navigation-3393400

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

ckrina created an issue. See original summary.

bnjmnm made their first commit to this issue’s fork.

finnsky’s picture

We also need:

1. Own testing tag EG: '@tags': ['navigation'],
2. Add tests in different themes contexts. Umami / Claro / Olivero / NO_THEME

Like was done https://git.drupalcode.org/project/navigation/-/merge_requests/79/diffs

bnjmnm’s picture

Status: Active » Needs review

The nightwatch MR has basic accessibility tests running. It's good to have these running, and this demonstrates that adding Nightwatch doesn't require any major errfor.

The Axe A11y tests already caught a duplicate ID, which is now fixed in that same MR. Given that this already caught an accessibility bug, I think this basic implementation should get in sooner than later so it can surface other accessibility problems as this project evolves.

The NW testing can get more comprehensive than what is currently there if we see the need. I think beginning with something simple like this is helpful as it gets basic A11y checks going and is a starting point for additional NW tests.

finnsky’s picture

Also creating some `dark` testing theme could be useful here.
Like:
body { background-color: black; color: white; }

ckrina’s picture

I wonder if #3402901: Toolbar doesn't indicate active menu trail for pages not included in Toolbar should be a classic test or a Nightwatch one.

finnsky changed the visibility of the branch nightwatch to hidden.

finnsky changed the visibility of the branch 3393400-evaluate-adding-nightwatch to hidden.

finnsky’s picture

Added a11y test and expand/collapse test.

Ally reports now

   ✖ 1) Tests/a11y

   – Accessibility - Navigation Module - Claro page (10.3s)

   → ✖ NightwatchAssertError
   aXe rule: button-name - Buttons must have discernible text
        In element: .toolbar-button--icon--sidebar-toggle
   → ✖ NightwatchAssertError
   aXe rule: link-name - Links must have discernible text
        In element: .toolbar-button--icon--system-themes-page
   → ✖ NightwatchAssertError
   aXe rule: link-name - Links must have discernible text
        In element: .toolbar-button--icon--system-modules-list
   → ✖ NightwatchAssertError
   aXe rule: link-name - Links must have discernible text
        In element: .toolbar-button--icon--entity-user-collection

   – Accessibility - Navigation Module - HomePage (6.613s)

   → ✖ NightwatchAssertError
   aXe rule: button-name - Buttons must have discernible text
        In element: .toolbar-button--icon--sidebar-toggle
   → ✖ NightwatchAssertError
   aXe rule: heading-order - Heading levels should only increase by one
        In element: .label
   → ✖ NightwatchAssertError
   aXe rule: landmark-unique - Ensures landmarks are unique
        In element: #admin-toolbar
   → ✖ NightwatchAssertError
   aXe rule: link-name - Links must have discernible text
        In element: .toolbar-button--icon--system-themes-page
   → ✖ NightwatchAssertError
   aXe rule: link-name - Links must have discernible text
        In element: .toolbar-button--icon--system-modules-list
   → ✖ NightwatchAssertError
   aXe rule: link-name - Links must have discernible text
        In element: .toolbar-button--icon--entity-user-collection
   → ✖ NightwatchAssertError
   aXe rule: region - All page content should be contained by landmarks
        In element: #primary-tabs-title

We need to add tooltip tests and mobile. basically will be fine

finnsky’s picture

hello all!
I have small misunderstanding.
We want to add Nightwatch testing. But i don’t really want to reproduce all Drupal Nightwatch infrastructure in module. Currently I’m running tests from core dir like yarn run test:nightwatch --tag navigation
But it will be useless in Navigation module CI.

So idk. I’ll continue in same approach. Probably we need to move Drupal Nightwatch functions into npm package. So it can be useful for modules/themes CI

Idk seems CI runned somehow. Need to check its context.

finnsky’s picture

Status: Needs review » Needs work
ckrina’s picture

Ideas on things that we could test:

  • Collapse/expand the sidebar
  • Scroll the sidebar (if the “content” is higher than the viewport height + footer)
  • Open/Close the drawer on hover (submenu level 2)
    • Test it with edge cases, like moving fast to the top of the drawer - so the mouse leaves the targeted hover item
  • Navigate to submenu level 3 (accordion)
  • Navigate to User menu links
  • See labels/tooltips of the 1st level links
  • Active menu item on 1st, 2nd and 3rd level items
  • (If logo) it takes you to the homepage
  • Active item shows as so in the Toolbar
finnsky’s picture

Issue summary: View changes
ckrina’s picture

Title: Evaluate adding Nightwatch tests » Implement Nightwatch tests
Issue summary: View changes

We've discussed this on several occasions and we've agreed on the need to implement these tests. So changing this from "evaluate" to "implement".

ckrina’s picture

Project: Navigation » Drupal core
Version: 1.x-dev » 11.x-dev
Component: Code » navigation.module
ckrina’s picture

Issue tags: +Portland2024

mglaman made their first commit to this issue’s fork.

mglaman’s picture

Started working on the tests, copied over @bnjmnm test https://git.drupalcode.org/project/drupal/-/merge_requests/7980

Experimental modules can't be installed using the install module command. since it won't be experimental for long, I just used a test setup script

mglaman’s picture

We have a problem here. The scroll wrapper, .admin-toolbar__scroll-wrapper, causes WebDriver to think any elements within it are not visible for interaction. Oddly enough waitForElementVisible passes. Probably because the latter is inspecting the DOM and not interacting with it.

When I made these changes, the tests passed:

.admin-toolbar__scroll-wrapper {
  /*display: flex;*/
  /*overflow-x: hidden;*/
  /*overflow-y: auto;*/
  flex-direction: column;
...
}

@media (min-width: 64rem) {
  .admin-toolbar__scroll-wrapper {
    /*display: contents;*/
    /*background: none;*/
  }
}

I also needed to fix the footer's z-index in #3446340: Fix z-index . Once the wrapper adjustments were made, the footer was hidden behind the navigation.

mglaman’s picture

Okay, I worked backward and added back styles. It is the display: contents

@media (min-width: 64rem) {
  .admin-toolbar__scroll-wrapper {
    display: contents;
mglaman’s picture

From caniuse https://caniuse.com/?search=display%3A%20contents

Buttons are not accessible with display: contents applied. See issues for Chromium, Firefox, and WebKit

mglaman’s picture

mglaman changed the visibility of the branch 3393400-nightwatch-tests to hidden.

mglaman’s picture

mglaman’s picture

Status: Needs work » Needs review
mglaman’s picture

Also pushed the axe test. It is failing. I think we should:

- Merge #3446357: Fix overflow visibility for wrapper content in navigation CSS to apply fix
- Follow @ckrina idea of making this a meta issue
- Commit the test site install script and expandCollapseTest.js (this issue or new child?)
- Make the child issues from #14 including a11y test

finnsky’s picture

I've added special tag for module for :

`yarn test:nightwatch --tag navigation`

finnsky’s picture

I've removed initial titles.
Now they are not defined.
Only block level titles we have now.

Backender review required here.

Test passed now.

finnsky’s picture

I think we can merge it and create META for other tests.

needs-review-queue-bot’s picture

Status: Needs review » Needs work
StatusFileSize
new90 bytes

The Needs Review Queue Bot tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".

This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

nod_’s picture

finnsky’s picture

Status: Needs work » Needs review
smustgrave’s picture

Status: Needs review » Needs work

Left a comment on MR. Should be small fix.

ckrina’s picture

We just defined an strategy to keep titles for the menus and avoid the problems with cache.

m4olivei made their first commit to this issue’s fork.

m4olivei’s picture

Status: Needs work » Needs review

Updated the merge request to address the <h4> issue. See MR comment for details.

plopesc’s picture

Status: Needs review » Needs work

Added comment suggesting a different approach to set the <h4> title using the original logic we had in place.
Let's decide the preferred approach and wrap this one!

m4olivei’s picture

Status: Needs work » Needs review

Addressed comments from @plopesc. Ready for review.

finnsky’s picture

Looks good to me. Also i've removed outdated classnames. they are only should be on block level now.

plopesc’s picture

Status: Needs review » Needs work

Thank you for the progress here!

I'm missing these 2 small tasks from the strategy defined in #37

  • Remove ID from h4and aria-labelledby from ul
  • Open a follow-up to add back the ID and aria-labelled-by with JS to avoid caching issues. Also in this follow-up replace the ID on the li items, r evaluate if they are needed.

Other than that, I think this is good to go!

finnsky’s picture

Status: Needs work » Needs review
finnsky’s picture

m4olivei’s picture

Status: Needs review » Reviewed & tested by the community

This is looking good to me! Items in #43 have been addressed. Tests are passing and making good sense to me.

finnsky’s picture

Sorry, keep in in rtbc. we also had that heading ID in footer.

nod_’s picture

Status: Reviewed & tested by the community » Needs work

few comments

plopesc’s picture

Status: Needs work » Reviewed & tested by the community

These changes are not affecting the code and tests are still green, so I'm moving it back to RTBC directly.

Thank you

nod_’s picture

Status: Reviewed & tested by the community » Needs work

committed followup, we can clean up this MR

plopesc’s picture

Status: Needs work » Reviewed & tested by the community

Back to RTBC once #3455058: drupalInstallModule nightwatch function does not work with Experimental modules was merged and the module installation process simplified.

nod_’s picture

Title: Implement Nightwatch tests » Implement Navigation Nightwatch tests

Thanks

nod_’s picture

Title: Implement Navigation Nightwatch tests » Implement Nightwatch tests for Navigation module

  • nod_ committed 74cd44af on 10.4.x
    Issue #3393400 by mglaman, finnsky, m4olivei, bnjmnm, plopesc, ckrina,...

  • nod_ committed 07d570cf on 11.0.x
    Issue #3393400 by mglaman, finnsky, m4olivei, bnjmnm, plopesc, ckrina,...

  • nod_ committed 9e83195c on 11.x
    Issue #3393400 by mglaman, finnsky, m4olivei, bnjmnm, plopesc, ckrina,...

nod_’s picture

Version: 11.x-dev » 10.4.x-dev
Category: Feature request » Task
Status: Reviewed & tested by the community » Fixed

not committing to 10.3.x because of the id removal, might backport once follow-up is in.

Committed and pushed 9e83195c5d to 11.x and 07d570cfc7 to 11.0.x and 74cd44af63 to 10.4.x. Thanks!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.