Extend DAST policy to support scheduled execution
Why are we doing this work
To support scheduled scans configured in Security Orchestration Policies we need to add workers and services to configure and execute scheduled jobs with selected scans.
Relevant links
Non-functional requirements
- [-] Documentation: added in separate issue,
-
Feature flag: all work will be hidden behind already created feature flag: security_orchestration_policies_configuration - [-] Performance: verify the impact of additional cron jobs on the system, coordinate with
CIteam, -
Testing: -
verify if pipelinescan policies are working properly, -
verify if schedulescan policies are working properly:-
check multiple cron expressions (with *,SUN,MON, numbers, etc.) -
check if jobs are properly scheduled when the repository with policies is updated,
-
-
Implementation plan
-
backend extend config/initializers/1_settings.rbwithSecurity::CreateOrchestrationPolicyWorkerandSecurity::OrchestrationPolicyRuleScheduleWorker, properly setcronvalue for these jobs,CreateOrchestrationPolicyWorkercan work every hour, butOrchestrationPolicyRuleScheduleWorkermust work quite often (ie. every 15 minutes) to make sure we can run scheduled scans on provided by users schedule (mention that in the documentation), -
backend extend ee/app/models/security/orchestration_policy_configuration.rbwith scope:with_outdated_configurationthat will get all policy configurations that does not have value forconfigured_atfield or for which value ofconfigured_atis earlier thansecurity_policy_management_project.last_activity_at-> this is not a perfect way to get information when was the last push tosecurity_policy_management_project(and potential policy update), but it will significantly reduce calls to database to createOrchestrationPolicyRuleScheduleinee/app/workers/security/create_orchestration_policy_worker.rb, -
backend extend ee/app/models/security/orchestration_policy_rule_schedule.rbwith scoperunnable_schedulesand methods needed bySchedulableconcern:ideal_next_run_from,cron_worker_next_run_fromandset_next_run_at;policymethod (to load policy configuration), -
backend add ee/app/workers/security/create_orchestration_policy_worker.rbthat will go throughSecurity::OrchestrationPolicyConfiguration.with_outdated_configurationin batches, and for each outdated configuration it will callSecurity::OrchestrationPolicies::ProcessRuleServiceservice, -
backend add ee/app/services/security/orchestration_policies/process_rule_service.rbthat will recreateSecurity::OrchestrationPolicyRuleSchedulein database for eachschedulepolicy and setconfigured_atforOrchestrationPolicyConfiguration, -
backend add ee/app/workers/security/orchestration_policy_rule_schedule_worker.rbthat will go throughSecurity::OrchestrationPolicyRuleSchedule.runnable_schedules(all schedules withnext_run_at < Time.now) in batches, and for each runnable schedule it will callSecurity::OrchestrationPolicies::RuleScheduleServiceservice, -
backend add ee/app/services/security/orchestration_policies/rule_schedule_service.rbthat will set new value fornext_run_atfield and go through actions in provided policy and start new pipeline with configured DAST On Demand Scan using::DastOnDemandScans::CreateService, when triggering a pipeline take last user that edited the policy file,
Edited by Sashi Kumar Kumaresan