Skip to content

Deduplicate scheduled AssignResourceFromResourceGroupWorker jobs

What does this MR do and why?

TLDR: this makes sure that scheduled AssignResourceFromResourceGroupWorker jobs (those called with perform_in instead of perform_async) still get deduplicated.

Further details

We recently introduced Re-spawn the AssignResource worker if busy (!147313 - merged) behind a Feature Flag (respawn_assign_resource_worker).

Before the respawn_assign_resource_worker was enabled, jobs in the pipelines often get stuck "waiting for resource" when there is more than one pipeline running at the same time. With the respawn_assign_resource_worker enabled, the jobs are no longer stuck with even up to 10 pipelines running at the same time.

However, looking at the logs, the re-spawned jobs don't seem to get deduplicated at all. While we don't expect re-spawned jobs to get deduplicated all the time, we still expect it to get deduplicated sometimes because it might "clash" with another AssignResourceFromResourceGroupWorker job that was, for example, queued through a state transition.

From the docs, scheduled jobs do not get automatically deduplicated. This MR makes sure that scheduled AssignResourceFromResourceGroupWorker jobs still get deduplicated.

Initial testing

The testing for the initial bug fix (Re-spawn the AssignResource worker if busy (!147313 - merged)) was done an example production group, where the pipeline structure is as follows:

  • there is one project (universal-rover) that serves as the "deployment project"
  • the universal-rover project triggers downstream pipelines in 10 different "application projects"
  • the 10 application projects each have their own deployment pipelines, with further child pipelines

Here is what a single pipeline in this example group looks like: https://gitlab.com/dkua1_ultimate_group/private/zd/gitlab-job-stuck-at-waiting-status-460524/universal-rover/-/pipelines/1248438407

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

N/A

How to set up and validate locally

Please see: !147313 (merged)

The Sidekiq jobs can be monitored on: <your-local-gitlab-url>/admin/background_jobs

Related to #436988 (closed)

Edited by Pam Artiaga

Merge request reports

Loading