Skip to content

Perform scheduled scan execution policy in a separate worker

Sashi Kumar Kumaresan requested to merge sk/411834-split-worker into master

What does this MR do and why?

Addresses #411834 (closed)

This MR introduces a new worker Security::ScanExecutionPolicies::RuleScheduleWorker to perform create pipelines from scan execution policy configured for a project from Security::OrchestrationPolicyRuleScheduleNamespaceWorker and Security::OrchestrationPolicyRuleScheduleWorker

The problem with the current approach is that the worker executed the logic to trigger pipelines for all projects in the same worker execution cycle. This creates problem if a single project has a failure causing the whole execution to fail. And retrial of failed jobs would cause duplicate pipelines created.

Current Flow

flowchart LR
    A(OrchestrationPolicyRuleScheduleWorker) --> |Project A| B[RuleScheduleService]
    B --> C[CreatePipelineService]
    A(OrchestrationPolicyRuleScheduleWorker) --> |Project B| D[RuleScheduleService]
    D --> E[CreatePipelineService]    
    A(OrchestrationPolicyRuleScheduleWorker) --> |Project C| F[RuleScheduleService]
    F --> G[CreatePipelineService]

New flow

flowchart LR
    K(OrchestrationPolicyRuleScheduleWorker) --> |Project A| A
    K(OrchestrationPolicyRuleScheduleWorker) --> |Project B| H
    K(OrchestrationPolicyRuleScheduleWorker) --> |Project C| I
    A(RuleScheduleWorker) --> |Project A| B[RuleScheduleService]
    B --> C[CreatePipelineService]
    H(RuleScheduleWorker) --> |Project B| D[RuleScheduleService]
    D --> E[CreatePipelineService]    
    I(RuleScheduleWorker) --> |Project C| F[RuleScheduleService]
    F --> G[CreatePipelineService]

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Sashi Kumar Kumaresan

Merge request reports