Add observability for parallelization inefficiencies
Context
- https://docs.google.com/document/d/1-puYWZtqHjqnAcGWc93KRlBE5Vnrky6lq2wH_tNV35s/edit
-
gitlab-org/gitlab#415160 (comment 1428431538)
- The pipeline is longer by 6 minutes because of a single RSpec job.
- Knapsack could have made a better job between the expected time a spec will take and the actual time it took
Goal
Show the gaps we might have between the expected time to run specs and the actual time.
The next goal would be to diagnose the root cause for those gaps, and try to make it more stable than it currently is.
Iteration proposal
In the job artifacts, we have an expected time for specs (coming from scheduled pipelines), and an actual time it took the particular job to run tests. Those files would need to be compared for potential mismatches.
- We could then receive an alert in Slack (might be very noisy), or at least display it somewhere.
Edited by David Dieulivol