Skip to content

Add prometheus metrics for scan execution policies

What does this MR do and why?

This MR adds a new prometheus metric security_policies_active_builds_scheduled_scans to inform the number of active ci builds created by scheduled scan execution policies.

Related to: #479218 (closed)

Database Query

SELECT
    COUNT(*)
FROM
    "p_ci_builds"
    INNER JOIN "ci_pipelines" "pipeline" ON "pipeline"."partition_id" IS NOT NULL
        AND "pipeline"."id" = "p_ci_builds"."commit_id"
        AND "pipeline"."partition_id" = "p_ci_builds"."partition_id"
WHERE
    "p_ci_builds"."type" = 'Ci::Build'
    AND "pipeline"."source" = 15
    AND ("p_ci_builds"."status" IN ('preparing', 'pending', 'running', 'waiting_for_callback', 'waiting_for_resource', 'canceling', 'created'))
    AND "p_ci_builds"."created_at" > '2024-08-20 10:18:29.671531'
    AND "p_ci_builds"."updated_at" > '2024-08-20 10:18:29.671601'

https://postgres.ai/console/gitlab/gitlab-production-ci/sessions/30890/commands/95951

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Screenshot_2024-08-19_at_11.25.16_AM

How to set up and validate locally

To validate this MR locally it is required to have the Prometheus and the sidekiq metric server enabled.

  1. Create a new project
  2. Get the project id
project_id = Project.last.id


 3. Add an empty test.rb file 4. Add an empty Gemfile.lock file 5. Go to Secure > Policies 6. Click in new policy 7. Select Scan Execution Policy 8. Change to the .yaml mode 9. Copy the policy content below

type: scan_execution_policy
name: policy
description: ''
enabled: true
policy_scope:
  projects:
    excluding: []
rules:
  - type: schedule
    cadence: 0 0 * * *
    timezone: Etc/UTC
    branch_type: all
actions:
  - scan: secret_detection
  - scan: sast
  - scan: sast_iac
  - scan: container_scanning
  - scan: dependency_scanning
  1. Click in configure with a merge request Merge the new MR

  2. Trigger the scheduled scans

  3. Get the schedule id in rails console

rule_schedule_id = Security::OrchestrationPolicyRuleSchedule.last.id
  1. Update the schedule next run_at to a time in the past using the gdk psql
UPDATE security_orchestration_policy_rule_schedules SET next_run_at = '2024-05-28 00:15:00+00' WHERE id = <rule_schedule_id>;
  1. Trigger the schedule in the rails console
Security::ScanExecutionPolicies::RuleScheduleWorker.perform_async(project_id, 1, rule_schedule_id)

Go to Build > Pipelines

Verify that a pipeline with 5 jobs was created 
8. Go to ‘:9090/graph’ and search for the security_policies_active_builds_scheduled_scans

Edited by Marcos Rocha

Merge request reports

Loading