Backport of 'Add concurrency limit to Security::Scans::PurgeByJobIdWorker' to 19.5
What does this MR do and why?
This MR backports !262567 (merged) to 19-5-stable-ee.
The MR is a git cherry-pick -x of the squash commit ab956650. The pick applied with no conflicts. The patch is identical to the default branch commit (same git patch-id).
What is backported
Security::Scans::PurgeByJobIdWorkergets an explicit concurrency limit of 25.- The worker removes duplicate job ids from the event before it calls
Security::PurgeScansService.purge_by_build_ids.
Why
The worker subscribes to Ci::JobArtifactsDeletedEvent. The artifact destroy path publishes one event for each batch of artifacts, and each event enqueues one purge job. A large destroy can enqueue many purge jobs at the same time. Without a limit, these jobs can hold many threads and database connections on the Sidekiq shard.
On self-managed and GitLab Dedicated, the default concurrency limit calculation is behind the sidekiq_concurrency_limit_default_calculation ops flag. This flag is off by default. So on 19.5 instances, the worker has no concurrency limit. This backport gives 19.5 instances the same limit as GitLab.com.
The related confidential issue https://gitlab.com/gitlab-org/gitlab/-/work_items/633838 has the details.
Risk
- The purge is idempotent. It only sets
security_scans.statustopurged. When the limit delays a job, no data is lost. - Jobs over the limit wait in the concurrency limit buffer.
ConcurrencyLimit::ResumeWorkerreleases them up to the limit. - To stop new jobs from going into the buffer, enable the
disable_sidekiq_concurrency_limit_middleware_Security::Scans::PurgeByJobIdWorkerfeature flag.
Default branch status
| Item | Value |
|---|---|
| Default branch MR | !262567 (merged) |
| Squash commit | ab956650 |
| Merged | 2026-10-20 |
| First deployed to GitLab.com production | 2026-10-20 |
Testing on 19-5-stable-ee
I ran these specs on the backport commit, on top of 19-5-stable-ee:
| Spec | Result |
|---|---|
ee/spec/workers/security/scans/purge_by_job_id_worker_spec.rb |
6 examples, 0 failures |
RuboCop found no offenses in the 2 changed files.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
- This MR is backporting a bug fix, documentation update, or spec fix, previously merged in the default branch.
- The MR that fixed the bug on the default branch has been deployed to GitLab.com (not applicable for documentation or spec changes).
- The MR title is descriptive (e.g. "Backport of 'title of default branch MR'"). This is important, since the title will be copied to the patch blog post.
- Required labels have been applied to this merge request
- severity label and bug subtype labels (if applicable)
- If this MR fixes a bug that affects customers, the customer label has been applied.
- This MR has been approved by a maintainer (only one approval is required).
- Ensure the
E2E Omnibus GitLab EEpipeline (triggered by thee2e:test-on-omnibus-eejob) has succeeded. If it has failed (or shows warnings), investigate the failures. If you determine the failures are unrelated, you may proceed. If you need assistance investigating, request help in the #s_developer_experience Slack channel to confirm the failures are unrelated to the merge request.
Note to the merge request author and maintainer
If you have questions about the patch release process, please:
- Refer to the patch release runbook for engineers and maintainers for guidance.
- Ask questions on the
#releasesSlack channel (internal only). - Once the backport has been merged, the commit changes will be automatically deployed to a release environment that can be used for manual validation. See after merging runbook for details.