Skip to content

Make ProcessScanResultPolicyWorker idempotent and add deduplication

Sashi Kumar Kumaresan requested to merge sk/393871-make-worker-idempotent into master

What does this MR do and why?

Addresses #393871 (closed)

Security::ProcessScanResultPolicyWorker is responsible for syncing scan result policies (that are stored in git repository) to create approval rules at project and merge request level.

graph LR
  Security::ProcessScanResultPolicyWorker --> id1([delete approval_rules and software_license_policies])
  Security::ProcessScanResultPolicyWorker --> id2([create approval_project_rules])
  Security::ProcessScanResultPolicyWorker --> id3([create approval_merge_request_rules])
  Security::ProcessScanResultPolicyWorker --> id5([create software_license_policies])
  Security::ProcessScanResultPolicyWorker --> Ci::SyncReportsToReportApprovalRulesWorker
  Ci::SyncReportsToReportApprovalRulesWorker --> id6([update approvals_required in mr approval rules])

The worker was not marked as idempotent but the functionality of the worker is idempotent as it deletes and re-creates the resources, the end state remains the same even it run multiple times. The worker should not execute concurrently for a same project_id and security_orchestration_policy_configuration_id as it deletes and re-creates resources. We were using ExclusiveLease module to achieve this, but we have that already at sidekiq level using until_executed.

This MR does 2 things:

  • Make the worker idempotent
  • Handle deduplication using until_executed

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Sashi Kumar Kumaresan

Merge request reports