Skip to content

Add time window for sep schedule scans

What does this MR do and why?

This MR adds a new property called time window for sep scheduled scans. This property is used to specify how the schedule scans should be distributed.

The current supported strategy is randomly scheduled the pipeline creation in the given time window.

References

Please include cross links to any resources that are relevant to this MR. This will give reviewers and future readers helpful context to give an efficient review of the changes introduced.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Screenshot_2024-12-03_at_5.33.02_PM

How to set up and validate locally

  1. Create a new group
  2. Create some projects and branches using the script
user = User.first
namespace_id = Group.last.id

10.times do
  project_params = {
    namespace_id: namespace_id,
    name: "Test-#{FFaker::Lorem.characters(15)}"
  }

  project = ::Projects::CreateService.new(user, project_params).execute
  project.save!

  project.repository.create_file(user, 'Gemfile.lock', '', branch_name: Gitlab::DefaultBranch.value,
    message: 'Add Gemfile.lock file')
  project.repository.create_file(user, 'test.rb', 'puts "hello world"', branch_name: Gitlab::DefaultBranch.value,
    message: 'Add test.rb file')

  6.times do
    branch_name = "branch-#{FFaker::Lorem.characters(15)}"
    ::Branches::CreateService.new(project, user).execute(branch_name, project.default_branch)
  end
end
  1. Go to the Group page
  2. Go to Secure > Policies
  3. Click in new policy
  4. Select Scan Execution Policy
  5. Change to the .yaml mode
  6. Copy the policy content below
type: scan_execution_policy
name: policy
description: ''
enabled: true
policy_scope:
  projects:
    excluding: []
rules:
  - type: schedule
    cadence: 0 0 * * *
    time_window:
      value: 3600
      distribution: random
    timezone: Etc/UTC
    branch_type: all
actions:
  - scan: secret_detection
  - scan: sast
  - scan: sast_iac
  - scan: container_scanning
  - scan: dependency_scanning
  1. Merge the policy

  2. Enable the feature flag scan_execution_pipeline_concurrency_control

Feature.enable(:scan_execution_pipeline_concurrency_control)
  1. Trigger the Security::SyncScanPoliciesWorker to create the OrchestrationPolicyRuleSchedule
Security::SyncScanPoliciesWorker.new.perform(Security::OrchestrationPolicyConfiguration.last.id)
  1. Get the schedule id in rails console
rule_schedule_id = Security::OrchestrationPolicyRuleSchedule.last.id
  1. Update the schedule next run_at to a time in the past using the gdk psql
UPDATE security_orchestration_policy_rule_schedules SET next_run_at = '2024-05-28 00:15:00+00' WHERE id = <rule_schedule_id>;
  1. Trigger the schedule in the rails console
Security::OrchestrationPolicyRuleScheduleNamespaceWorker.new.perform(rule_schedule_id)
  1. Go the Admin Area > Monitoring > Background jobs

  2. Click on the scheduled tab

  3. Click on Stop Polling

  4. Filter by Security::ScanExecutionPolicies::CreatePipelineWorker and verify the jobs are scheduled randomly in the time window

group = Group.last

project_ids = group.all_projects.map(&:id)

Ci::Pipeline.where(source: 'security_orchestration_policy').where(project_id: project_ids).group("date_trunc('minute', created_at)").count
Edited by Marcos Rocha

Merge request reports

Loading