Skip to content

Extend DAST policy to support scheduled execution

Why are we doing this work

To support scheduled scans configured in Security Orchestration Policies we need to add workers and services to configure and execute scheduled jobs with selected scans.

Relevant links

Non-functional requirements

  • [-] Documentation: added in separate issue,
  • Feature flag: all work will be hidden behind already created feature flag: security_orchestration_policies_configuration
  • [-] Performance: verify the impact of additional cron jobs on the system, coordinate with CI team,
  • Testing:
    • verify if pipeline scan policies are working properly,
    • verify if schedule scan policies are working properly:
      • check multiple cron expressions (with *, SUN,MON, numbers, etc.)
      • check if jobs are properly scheduled when the repository with policies is updated,

Implementation plan

  • backend extend config/initializers/1_settings.rb with Security::CreateOrchestrationPolicyWorker and Security::OrchestrationPolicyRuleScheduleWorker, properly set cron value for these jobs, CreateOrchestrationPolicyWorker can work every hour, but OrchestrationPolicyRuleScheduleWorker must work quite often (ie. every 15 minutes) to make sure we can run scheduled scans on provided by users schedule (mention that in the documentation),
  • backend extend ee/app/models/security/orchestration_policy_configuration.rb with scope :with_outdated_configuration that will get all policy configurations that does not have value for configured_at field or for which value of configured_at is earlier than security_policy_management_project.last_activity_at -> this is not a perfect way to get information when was the last push to security_policy_management_project (and potential policy update), but it will significantly reduce calls to database to create OrchestrationPolicyRuleSchedule in ee/app/workers/security/create_orchestration_policy_worker.rb,
  • backend extend ee/app/models/security/orchestration_policy_rule_schedule.rb with scope runnable_schedules and methods needed by Schedulable concern: ideal_next_run_from, cron_worker_next_run_from and set_next_run_at; policy method (to load policy configuration),
  • backend add ee/app/workers/security/create_orchestration_policy_worker.rb that will go through Security::OrchestrationPolicyConfiguration.with_outdated_configuration in batches, and for each outdated configuration it will call Security::OrchestrationPolicies::ProcessRuleService service,
  • backend add ee/app/services/security/orchestration_policies/process_rule_service.rb that will recreate Security::OrchestrationPolicyRuleSchedule in database for each schedule policy and set configured_at for OrchestrationPolicyConfiguration,
  • backend add ee/app/workers/security/orchestration_policy_rule_schedule_worker.rb that will go through Security::OrchestrationPolicyRuleSchedule.runnable_schedules (all schedules with next_run_at < Time.now) in batches, and for each runnable schedule it will call Security::OrchestrationPolicies::RuleScheduleService service,
  • backend add ee/app/services/security/orchestration_policies/rule_schedule_service.rb that will set new value for next_run_at field and go through actions in provided policy and start new pipeline with configured DAST On Demand Scan using ::DastOnDemandScans::CreateService, when triggering a pipeline take last user that edited the policy file,
Edited by Sashi Kumar Kumaresan