Add time window for sep schedule scans
What does this MR do and why?
This MR adds a new property called time window for sep scheduled scans. This property is used to specify how the schedule scans should be distributed.
The current supported strategy is randomly scheduled the pipeline creation in the given time window.
References
Please include cross links to any resources that are relevant to this MR. This will give reviewers and future readers helpful context to give an efficient review of the changes introduced.
- Related to #509137 (closed)
MR acceptance checklist
Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Screenshots or screen recordings
How to set up and validate locally
- Create a new group
- Create some projects and branches using the script
user = User.first
namespace_id = Group.last.id
10.times do
project_params = {
namespace_id: namespace_id,
name: "Test-#{FFaker::Lorem.characters(15)}"
}
project = ::Projects::CreateService.new(user, project_params).execute
project.save!
project.repository.create_file(user, 'Gemfile.lock', '', branch_name: Gitlab::DefaultBranch.value,
message: 'Add Gemfile.lock file')
project.repository.create_file(user, 'test.rb', 'puts "hello world"', branch_name: Gitlab::DefaultBranch.value,
message: 'Add test.rb file')
6.times do
branch_name = "branch-#{FFaker::Lorem.characters(15)}"
::Branches::CreateService.new(project, user).execute(branch_name, project.default_branch)
end
end
- Go to the Group page
- Go to Secure > Policies
- Click in new policy
- Select Scan Execution Policy
- Change to the .yaml mode
- Copy the policy content below
type: scan_execution_policy
name: policy
description: ''
enabled: true
policy_scope:
projects:
excluding: []
rules:
- type: schedule
cadence: 0 0 * * *
time_window:
value: 3600
distribution: random
timezone: Etc/UTC
branch_type: all
actions:
- scan: secret_detection
- scan: sast
- scan: sast_iac
- scan: container_scanning
- scan: dependency_scanning
-
Merge the policy
-
Enable the feature flag
scan_execution_pipeline_concurrency_control
Feature.enable(:scan_execution_pipeline_concurrency_control)
- Trigger the
Security::SyncScanPoliciesWorker
to create theOrchestrationPolicyRuleSchedule
Security::SyncScanPoliciesWorker.new.perform(Security::OrchestrationPolicyConfiguration.last.id)
- Get the schedule id in rails console
rule_schedule_id = Security::OrchestrationPolicyRuleSchedule.last.id
- Update the schedule next
run_at
to a time in the past using thegdk psql
UPDATE security_orchestration_policy_rule_schedules SET next_run_at = '2024-05-28 00:15:00+00' WHERE id = <rule_schedule_id>;
- Trigger the schedule in the rails console
Security::OrchestrationPolicyRuleScheduleNamespaceWorker.new.perform(rule_schedule_id)
-
Go the Admin Area > Monitoring > Background jobs
-
Click on the scheduled tab
-
Click on Stop Polling
-
Filter by Security::ScanExecutionPolicies::CreatePipelineWorker and verify the jobs are scheduled randomly in the time window
group = Group.last
project_ids = group.all_projects.map(&:id)
Ci::Pipeline.where(source: 'security_orchestration_policy').where(project_id: project_ids).group("date_trunc('minute', created_at)").count