Add JSON reporter to produced artifacts in Automated Accessibility
Problem to solve
The a11y MVC is great in giving me a viewable HTML artifact, but it's huge and not easy to digest which issues are the worst and which category of issues are most prevalent. Beyond that everyone who wants to see this has to download it and follow instructions about how to view the file and get the data. If I could see the latest accessibility report for my project in GitLab it would be much more valuable for me to create new issues to improve the a11y of my project for users.
One of the produced artifacts should be a JSON file with all the results from the accessibility scan.
Intended users
A developer who wants to consume a11y data for their repository branches and merge requests, including our own frontend devs in 12.10: #39425 (closed)
Further details
Proposal
Run a pa11y scan with the reporter set to json as well as html The simplest implementation of this will be to run the pa11y scan twice, which could be inefficient for a large number of pages but is the MVC that produces the data we want.
We currently use the plain pa11y library, but pa11y-ci is available and easy to use as well.
Permissions and Security
Documentation
Testing
What does success look like, and how can we measure that?
What is the type of buyer?
Links / references
This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.
Designs
Relates to
Activity
changed milestone to %12.8
added to epic &2021 (closed)
added [deprecated] Accepting merge requests label
changed milestone to %Backlog
Moved to backlog until we get through solution validation for this to know which direction is first cc @jj-ramirez
added workflowstart label and removed workflowplanning breakdown label
mentioned in issue #207959 (closed)
marked #207959 (closed) as a duplicate of this issue
marked this issue as related to #207959 (closed)
added featureenhancement label
changed milestone to %12.9
assigned to @drewcimino
@jheimbuck_gl Heads up that I'm removing GitLab Premium and moving this into 12.9
removed GitLab Premium label
removed [deprecated] Accepting merge requests label
added workflowready for development label and removed workflowstart label
added workflowin dev label and removed workflowready for development label
I'm marking this as ~"workflow::In dev" since I already have the template to produce JSON artifacts worked out https://gitlab.slack.com/archives/CPANF553J/p1582647319012000 (I'll update this link when the repository is up)
Pushing to the a11y repository and once the image is published to the Gitlab container registry, I can open an a11y template MR to run
pa11y-ci --jsonwith CHANGED_PAGES variables !25256 (merged)Edited by drew stachonmentioned in issue #39425 (closed)
Here's the WIP MR: !26413 (closed)
It depends on the
accessibilityimage, so I haven't run the CI yet.added workflowin review label and removed workflowin dev label
added workflowverification label and removed workflowin review label
Verified in 12.9 https://gitlab.com/drewcimino/a11y-scanning/-/jobs/475297041
removed workflowverification label
added UX scorecard-rec label
added Category:Code Testing and Coverage testingaccessibility labels