Create annual GitLab CI job performance benchmark report for SaaS solutions.
Overview
Ensuring that users and customers can efficiently execute CI jobs on the GitLab SaaS Runners is critical to the GitLab product strategy. From time to time, we get requests from customers, sales, or members of the leadership team regarding CI jobs' performance data on GitLab SaaS compared to other solutions in the marketplace.
- In this issue, we established a performance and load testbed for GitLab Runner.
- In this issue, we performed some basic performance comparisons of CI jobs on GitLab SaaS runners to other solutions in the market.
As a follow on to that work, we believe there is an opportunity for GitLab to show thought leadership in this space, i.e., CI job performance on hosted Value Stream Delivery platforms. Of course, CI job performance is impacted by multiple variables, including but not limited to compute architecture, project size, CI job composition.
Proposal
Create an annual GitLab CI job performance benchmark report. The inspiration for the report and format is based on the work done by Cockroach labs in their 2021 Cloud Report.
Link to report here.
This is a great idea! Could be a cool thing to plan into a specific release- such as the .0 release every year (ie: time it with 14.0, 15.0).
For the MVC, the first report should be focused on a few specific use cases.