Automated a11y scanning of Review Apps
Problem to solve
Performing accessibility testing is important in order to ensure you're serving all the users who use your products. We offer code quality validation automatically via codeclimate by scanning code at build time, we could also provide a comprehensive accessibility scan by running https://github.com/liip/TheA11yMachine or https://github.com/pa11y/pa11y-ci against a built web application as deployed to a review app.
Proposal
Upon successful deployment of a review app, use one of the a11y scanning tools and the route map (if necessary) to scan the review app for accessibility issues, which can be reported as a CI artifact and/or in the MR view.
As an MVC modeled after performance testing, use the container image @brendan built here: https://gitlab.com/gitlab-org/gl-a11ym to run the test and output the report. An example of this can be found here: https://gitlab.com/brendan-demo/a11ym-testing
What does success look like, and how can we measure that?
- Have CE/EE merge requests use this from the start instead of implementing a separate tooling for a11y testing.
- Identify accessibility improvements that were discovered as part of this new auto scanning step
- Also like to expand this to other projects e.g. GitLab insights.