Add runtime limiter to ProjectComplianceEvaluatorWorker
What does this MR do and why?
Adds a runtime limiter to ComplianceManagement::ProjectComplianceEvaluatorWorker so it stays under the 300s Sidekiq SLO for :low urgency workers. This worker is breaching that SLO (~14372 overruns).
Root cause: the fan-out caller ComplianceManagement::ComplianceFramework::ProjectsComplianceEnqueueWorker batches at each_batch(of: 100), so a single run processes up to 100 projects × all requirements × all controls, each doing DB writes and possibly an outbound HTTP call (TriggerExternalControlService).
Fix (smallest change addressing the root cause):
- Wrap the
projects.eachloop withGitlab::Metrics::RuntimeLimiter, checkingover_time?at the project loop boundary so each project's evaluation stays atomic (the limiter never trips mid-requirement/mid-control). - When over time, re-enqueue the unprocessed
project_idsviaself.class.schedule_compliance_evaluation(framework_id, remaining)andbreak. - Use the default
RuntimeLimiterbudget (200s), which sits under the 300s SLO. - Existing behaviour is preserved: the
framework.project_idsintersection, per-control rescue/error tracking, andupdate_requirement_statuscalls are unchanged.
Dedup safety: the worker uses deduplicate :until_executed, including_scheduled: true with no idempotency_arguments override, so the dedup key is built from the full [framework_id, project_ids] argument array. A re-enqueue with the remaining subset is a distinct key and is not dropped. No override added.
Tests: added specs covering the limiter tripping mid-loop, the correct remaining project_ids being re-enqueued, and no re-enqueue when all projects complete.
References
- Resolves #630492
Screenshots or screen recordings
N/A — backend worker change, no UI.
How to set up and validate locally
- Run the worker spec:
bundle exec rspec ee/spec/workers/compliance_management/project_compliance_evaluator_worker_spec.rb - New examples cover: limiter tripping mid-loop (stops after the in-flight project and re-enqueues the remaining ids), the exact remaining
project_idsre-enqueued, and no re-enqueue when all projects are evaluated.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.