Move CancelPendingPipelines step to run async
Problem
The step CancelPendingPipeline
of the pipeline creation ranges from 6s to 19s. This time is spent looking for previous pipelines that are pending or running on the same ref where the new pipeline has just been created. Then cancel these pipelines one by one, including any child pipelines.
-
gitlab-org/gitlab
project - https://log.gprd.gitlab.net/goto/5bb29a20-6038-11ed-b0ec-930003e0679c (stacked bar of all creation steps) - all projects - https://log.gprd.gitlab.net/goto/d0a7be00-698e-11ed-9f43-e3784d7fe3ca (stacked bar of all creation steps)
This is something we should be able to do asynchronously.
Solution
Move the logic to a Sidekiq worker and ensure that the CancelPendingPipeline
step duration is reduced to nearly 0s.
Expected benefit
15-20% reduction in pipeline creation time.
Edited by Fabio Pitino