Performance regression testing framework
Problem statement
Today at GitLab, we have little to no way of knowing if a Merge Request has negatively impacted performance. This leads to a reactive approach, and discovering performance problems in production. Relying on our customers to detect performance problems is less than ideal.
Solution
We need to develop a performance regression test suite, where each MR is tested to determine if it negatively impacts broader GitLab performance. This way the developer, and reviewer, can have a better understanding of whether a given change is ready to be merged.
We would want to:
- Have a repeatable way to generate load across a variety of endpoints
- Monitor and analyze system metrics like CPU, Memory, Disk, I/O, etc.
- Monitor and analyze external service usage like DB and Redis.
- Monitor and analyze browser performance, like latency.
Potential utilization of GitLab features
GitLab has two relevant features:
- Load Testing: &952 (closed)
- Planned in 2020
- Browser Testing: https://docs.gitlab.com/ee/user/project/merge_requests/browser_performance_testing.html
Not specifically a GitLab feature, but we did build load testing scripts for our Reference Architecture work which could be leveraged.
Edited by Joshua Lambert