Reduce pipeline cost by reducing number of rspec tests before MR approval
Objective
Reduce the CI cost for GitLab pipelines by running the most applicable rspec tests for changes prior to approval.
Approach
Tests will be executed in merge request pipelines that are related to the changed files in a merge request prior to the first approval. These tests will be selected based on dynamic mapping and tests that are modified within the MR.
Engineering productivity believes this will lead to:
- Faster feedback from merge request pipelines
- Reduction of CI costs on the GitLab project
Plan
- Incrementally release MR prompt to start a new pipeline after approval and gather feedback:
-
2021-07-19: roll out to 25% of MRs -
2021-07-20: roll out to 50% of MRs -
2021-07-21: roll out to 100%
-
-
2021-07-26: Merge CI config change.
Risk
We have identified two risks and a mitigation for each risk.
Risk | Mitigation |
---|---|
Merge requests that are merged with only a minimal pipeline can break master. Early experiments indicated this could happen 10% of the time | Run the full tests after first approval to improve the mapping over time. |
Merge requests do not have a pipeline triggered after the first approval 30-40% of the time | Add a nudge to trigger a full pipeline after approval |
Feedback
For feedback on either the prompt or the pipeline changes, please comment on this issue.
Edited by Rémy Coutable