Process data in configuration repository for DAST ScanExecutionPolicy

Why are we doing this work

As mentioned in &4598 (comment 491595129) (1A) we want to implement MVC using GitLab repositories to store security policies. We want to be able to parse and act based on actions and rules defined in YAML policy file. This issue addresses that requirement.

This issue we will implement new background workers and services to store data that is required to perform scheduled DAST scans.

Type of services/workers will be all services related to calculating time of next scan execution (similar to all services related to PipelineSchedule model (more details)).

Relevant links

Implementation plan

  • Changes will be behind security_orchestration_policies feature flag
  • backend Implement cron worker that will parse policies defined in repository and will prepare models in database for scheduled actions (SecurityOrchestrationPolicyRuleSchedule),
    • backend Make sure this worker iterates on each SecurityOrchestrationPolicyConfiguration (each pair of ProjectID and SecurityOrchestrationPolicyProjectID) and for each active policy (for each YAML policy file within repository) initiate new Service (ProcessRuleService) that will take care of:
      • making sure we do not have records in database that are not defined in policies,
      • making sure we have records in database for SecurityOrchestrationPolicyRuleSchedule and creating them when policy has changed (we should make sure we have next_run_at set for this model to be able to properly schedule it),
    • backend initially set this worker to run every 30 minutes
Edited by Adam Cohen