Knapsack - Ensure the RSpec jobs are evenly distributed
Context
- https://docs.google.com/document/d/1-puYWZtqHjqnAcGWc93KRlBE5Vnrky6lq2wH_tNV35s/edit
-
gitlab-org/gitlab#415160 (comment 1428431538)
- The pipeline is longer by 6 minutes because of a single RSpec job.
- Knapsack could have made a better job between the expected time a spec will take and the actual time it took
Goal
- Understand whether we could improve Knapsack load distribution for RSpec jobs, so that jobs finish around the same time.
How to measure that we improved the situation?
To be able to measure whether we were successful in distributing the load more evenly for Knapsack, we created #241 (closed) to add observability to the process.
Technical hints
- Knapsack use a so called "master report" for scheduling RSpec files to jobs. This file is generated every 2 hours in scheduled pipelines.
- It is possible that this file is subject to variability (e.g. slow runner, slower tests due to initial setup, ...). To reduce this variability, we might want to generate this master report with 3-5 previous master reports, and average them:
- create
report-master-1.json
,report-master-2.json
,report-master-3.json
- merge them
- have one
report-master.json
to be used in future jobs)
- create
- It is possible that this file is subject to variability (e.g. slow runner, slower tests due to initial setup, ...). To reduce this variability, we might want to generate this master report with 3-5 previous master reports, and average them:
Edited by David Dieulivol