Estimate the costs of E2E test parallelization
Given that parallelization is likely to increase our usage of CI resources, and therefore increase our costs, it would be prudent to understand our current costs and to estimate how much parallelization will increase those costs.
For a very rough estimate, assuming review-qa-all
runs once per pipeline, in 8000 pipelines per month, at a cost of $0.0950 per hour, with a job execution time of 7m 30s (not parallel) or 17m (parallel: 5 jobs), the cost per month would be:
- not parallel: $95
- parallel: $215
That extra cost would save us about 3m 30s in each pipeline (because the instead of waiting 7m 30s for the job to complete, we'd wait only the duration of the slowest parallel job, which is about 4m).
Note: this estimate only considers parallelizing the review-qa-all
job, and assumes it will run in every MR. We should also consider the cost of parallelizing package-and-qa
This script could be useful: https://gitlab.com/gitlab-org/quality/toolbox#gather-jobs-durations