Problem/Motivation

So far, we have very little functional test coverage for keyboard operations, distinct from pointer operations.

Prior to using WebDriver this was awkward anyway, because Goutte keyboard driving was flaky.

In the last couple of years we have started to add some keyboard test coverage:

We now have some significant experience of writing functional tests for keyboard operations. So let's start expanding our keyboard accessiblility test coverage...

Proposed resolution

1) proposed text
Add new heading to the core testing gate

Add Keyboard operation test

When is this needed?
  1. When adding a new UI feature.
  2. Changes that involves the JS keyboard, focus, or blur events.
Details
  • Ensure compliance with {TBA}
Resources

{Add link to d.o doc about how to write such a test}

2) Create or add the following to a d.o doc about testing

Things we can write keyboard behaviour tests for

Here's an initial survey to find testable keyboard behaviours.

  1. "Mobile menu" behaviour at small breakpoints, e.g. in Bartik and Umami themes. Is the menu operable with a keyboard?
    • Assert: the menu button opens and closes using either the enter or spacebar keys.
    • Assert: the nested menu items are NOT in the keyboard tabbing order when the menu is closed. If they are, that's a failure of WCAG "Focus visible".
  2. When a dialog, opens, focus moves inside it.
  3. Dialogs can be dismissed by pressing ESC.
  4. When a dialog is dismissed, focus returns to the button that opened it. May need separate tests for each place that dialogs are used.
  5. Assert that the colllapse.js details works with spacebar and enter.
  6. Assert that keyboard focus does not move to operable controls which are concealed inside containers which are in a closed state. For example only controls in the active vertical tab can be tabbed to, not controls in the other tabs.
  7. The CKEditor toolbar configuration UI. Lots of interesting scenarios there.

You can tag issues with "needs accessibility review" to get help with devising keyboard scenarios and/or clarify expected behaviour.

Remaining tasks

  1. Catalogue existing keyboard operations we can add test scenarios for. In progress, forming a list under a separate heading above.
  2. There is a manual test plan somewhere. Are there any keyboard related things there?
  3. Find the existing issue about converting the manual test plan to Nightwatch.

Comments

andrewmacpherson created an issue. See original summary.

andrewmacpherson’s picture

Issue tags: +Testing system, +test coverage.

Initial brain dump, please discuss :-)
I'm not sure what the best component is, or the preferred topic tags for testing-related issues

andrewmacpherson’s picture

Issue summary: View changes

Added a few example assertions about keyboard behaviour we could write.

andrewmacpherson’s picture

We can also include assertions that dynamic aria-* properties are correctly updated to reflect the UI state, e.g. sortable table columns. Some of this might already have Functional JS tests, when testing with click(). It would be good to include these assertions in the keyboard tests too, in case we have any aria-* properties which are updated by mouse* event handlers, and omitted from key* handlers.

andrewmacpherson’s picture

Issue summary: View changes
Issue tags: -Accessibility +accessibility
tim.plunkett’s picture

Issue tags: -accessibility (duplicate tag), -test coverage. +Accessibility

Fixing tags

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

mgifford’s picture

Issue tags: -JavaScript +JavaScript

I like this idea. With automated tools and simple keyboard-only testing we can catch the vast majority of errors.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

quietone’s picture

Title: [POLICY; NO PATCH] Require functional test coverage for keyboard accessibility » [policy, no patch] Require functional test coverage for keyboard accessibility

Just changing title per Special titles.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

quietone’s picture

Issue summary: View changes
Status: Active » Needs work

Updated the IS with proposed text for an addition to the core testing gate. It needs work for details so changing status.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.