Skip to content

Add time_window for scan execution policies

In the scope of this issue we want tot add a new property called time_window for scan execution policy scheduled scans.

This property will be used to specify how the schedule scans should be distributed.

The strategy implemented in this issue is to randomly scheduled the pipeline creation in the given time_window.

How to set up and validate locally

  1. Create a new group
  2. Create some projects and branches within the group
  3. Go to the Group page
  4. Go to Secure > Policies
  5. Click in new policy
  6. Select Scan Execution Policy
  7. Change to the .yaml mode
  8. Copy the policy content below
type: scan_execution_policy
name: policy
description: ''
enabled: true
policy_scope:
  projects:
    excluding: []
rules:
  - type: schedule
    cadence: 0 0 * * *
    time_window:
      value: 3600
      distribution: random
    timezone: Etc/UTC
    branch_type: all
actions:
  - scan: secret_detection
  - scan: sast
  - scan: sast_iac
  - scan: container_scanning
  - scan: dependency_scanning

  1. Merge the policy

  2. Enable the feature flag scan_execution_pipeline_concurrency_control

Feature.enable(:scan_execution_pipeline_concurrency_control)

  1. Wait for the scheduled time

  2. Go to /-/graphql-explorer

  3. Query the pipeline creation times for the projects in the group using a query like

{
  group(
    fullPath: "gitlab-org/security-risk-management/security-policies/team-member-environment/marcos-test-group/test-scheduled-scans-new-worker"
  ) {
    projects {
      nodes {
        name
        pipelines(updatedAfter: "2024-12-05T18:28:38Z") {
          nodes {
            id
            createdAt
            ref
          }
          count
          pageInfo {
            hasNextPage
          }
        }
      }
    }
  }
}

Or get the times of the pipeline creation for a single project using the query:

{
  project(
    fullPath: "gitlab-org/security-risk-management/security-policies/team-member-environment/marcos-test-group/test-scheduled-scans-new-worker/project-2"
  ) {
    pipelines(updatedAfter: "2024-12-05T18:28:38Z") {
      nodes {
        id
        createdAt
        ref
      }
      count
      pageInfo {
        hasNextPage
      }
    }
  }
}

You can copy the results of the above query to a file called data.json and use a script like the one below to get the number of pipelines created by minute.

require 'json'
require 'time'

data_json = JSON.load_file('data.json')
data_hash = data_json

pipelines = data_hash.dig('data', 'project', 'pipelines', 'nodes')

pipeline_creation_times = pipelines.map do |pipeline|
  Time.strptime(pipeline.dig('createdAt'), '%Y-%m-%dT%H:%M:%S')
end


puts pipeline_creation_times.sort.join("\n")

pipeline_creation_times = pipelines.map do |pipeline|
  Time.strptime(pipeline.dig('createdAt'), '%Y-%m-%dT%H:%M')
end

puts pipeline_creation_times.sort.tally
Edited by Marcos Rocha