POC: visual testing with Percy
The follow up of a quick spike done in 12.9 for the issue Create e2e (or integration) visual test for the metrics dashboard (#205631)
There are several tools that can be leveraging visual testing, being Applitools and Percy some of the most known options.
Percy as a proof of concept:
- Integrates with GitLab CI - we will be able to understand how the visual tests integrate with MRs
- We are already using Capybara on GitLab QA specs and Percy has great documentation on how to easily integrate it
- Percy’s free plan includes 5,000 free monthly screenshots, 10 users, and unlimited projects, which is more than enough for us to start with
- Intelligentbase build selection and screenshot stabilization strategies can mitigate the usual hassle of maintaining visual tests (this is a first assumption to be tested while building the POC)
- Findings with Percy can be useful to do a final comparison on adoption of a visual testing tool
The MVC:
The bug Charts with series that contain multiple labels do not have unique colors for each label can be the first scenario that this POC can test. This is also useful for regressions on graphs.
After assessing the workflow and ease of integration with this MVC, the use of Percy could be assessed and dogfooded with developers.