It would be really great if there was a simple process to extend core's tests for use in a contrib module to ensure that all of the accessibility improvements are not accidentally broken by the contrib module.

Initial idea:

  • Refactor / streamline / simplify the tests around accessibility so that they have a hierarchical structure:
    • Base tests for system output.
    • Each bundled theme would extend these to include any additional accessibility adjustments necessary for the theme, e.g. DOM-level adjustments where the output adds extra DIVs, etc.
  • Clearly document how a contrib theme would go about leveraging the core tests.
  • Later on.. create a generator in DrupalConsole to create the necessary test files for a given subtheme.

Comments

DamienMcKenna created an issue. See original summary.

mgifford’s picture

mgifford
he/him
Primary language English
commented

There was some automated testing for accessibility that @jessebeach set up with QuailJS. It was somehow tied to Test Swarm. Anyways, not sure what happened with that, but really don't think it's been working for years. Was great when it did. Allowed us to see if there were more or less errors in Core from one day to the next.

Right now though we don't have an automated accessibility testing infrastructure set up. There are a lot of hopeful options though, like:

https://github.com/dequelabs/axe-core
https://tenon.io/
http://quailjs.org/
http://pa11y.org/
https://github.com/Tanaguru/Tanaguru
https://github.com/crawlkit

Just need someone (or a group of folks) to take this on.

andrewmacpherson’s picture

andrewmacpherson commented

There is also http://asqatasun.org/, which I think replaces Tanaguru - I'm not sure if it's a fork or a successor exactly.

andrewmacpherson’s picture

andrewmacpherson commented

Note that when #2731373: [upstream] Include an accessibility tester in CKEditor (axe, Tota11y, HTML Code Sniffer) is fixed, it will mean QuailJS is bundled with Drupal core.

mgifford’s picture

mgifford
he/him
Primary language English
commented

Sorry to read (in that Asqatasun page) that 'the start-up did not succeed in building a sustainable business and collapsed in 2014, loosing the brand "Tanaguru"'.

xjm’s picture

xjm
she/her
Primary language English
commented
Version: 8.3.x-dev » 8.2.x-dev

This sounds like a really great idea!

Since it sounds like these will mostly be testing improvements, they should be eligible to be added during patch releases. Some API additions etc. might be targeted for the minor instead, depending on the issue.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.6 was released on August 2, 2017 and is the final full bugfix release for the Drupal 8.3.x series. Drupal 8.3.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.4.0 on October 4, 2017. (Drupal 8.4.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.4.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.4 was released on January 3, 2018 and is the final full bugfix release for the Drupal 8.4.x series. Drupal 8.4.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.5.0 on March 7, 2018. (Drupal 8.5.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.5.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

mgifford’s picture

mgifford
he/him
Primary language English
commented
Parent issue: » #2857808: Automate Accessibility Checks for Core

Testing is done in https://www.drupal.org/project/drupalci so setting parent issue.

mgifford’s picture

mgifford
he/him
Primary language English
commented
Version: 8.5.x-dev » 8.6.x-dev

Could be a matter of simply spinning up the default theme and running axe-core on the home page with something like https://humanmade.com/2018/05/17/automated-accessibility-testing-during-...

Hopefully we can build on some of the work that the WP community has started.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

mgifford’s picture

mgifford
he/him
Primary language English
commented
Related issues: +#3293469: Automated A11y tests in Nightwatch, +#3338664: Migrate Nightwatch Axe tests to PHPUnit

Now we have nightwatch-axe in core, we're a step ahead. Also tagging for PHPUnit - axe which was flagged recently too.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.