Add prometheus metrics for scan execution policies
What does this MR do and why?
This MR adds a new prometheus metric security_policies_active_builds_scheduled_scans
to inform the number of active ci builds created by scheduled scan execution policies.
Related to: #479218 (closed)
Database Query
SELECT
COUNT(*)
FROM
"p_ci_builds"
INNER JOIN "ci_pipelines" "pipeline" ON "pipeline"."partition_id" IS NOT NULL
AND "pipeline"."id" = "p_ci_builds"."commit_id"
AND "pipeline"."partition_id" = "p_ci_builds"."partition_id"
WHERE
"p_ci_builds"."type" = 'Ci::Build'
AND "pipeline"."source" = 15
AND ("p_ci_builds"."status" IN ('preparing', 'pending', 'running', 'waiting_for_callback', 'waiting_for_resource', 'canceling', 'created'))
AND "p_ci_builds"."created_at" > '2024-08-20 10:18:29.671531'
AND "p_ci_builds"."updated_at" > '2024-08-20 10:18:29.671601'
https://postgres.ai/console/gitlab/gitlab-production-ci/sessions/30890/commands/95951
MR acceptance checklist
Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Screenshots or screen recordings
How to set up and validate locally
To validate this MR locally it is required to have the Prometheus and the sidekiq metric server enabled.
- Create a new project
- Get the project id
project_id = Project.last.id
3. Add an empty test.rb file 4. Add an empty Gemfile.lock file 5. Go to Secure > Policies 6. Click in new policy 7. Select Scan Execution Policy 8. Change to the .yaml mode 9. Copy the policy content below
type: scan_execution_policy
name: policy
description: ''
enabled: true
policy_scope:
projects:
excluding: []
rules:
- type: schedule
cadence: 0 0 * * *
timezone: Etc/UTC
branch_type: all
actions:
- scan: secret_detection
- scan: sast
- scan: sast_iac
- scan: container_scanning
- scan: dependency_scanning
-
Click in configure with a merge request Merge the new MR
-
Trigger the scheduled scans
-
Get the schedule id in rails console
rule_schedule_id = Security::OrchestrationPolicyRuleSchedule.last.id
- Update the schedule next run_at to a time in the past using the gdk psql
UPDATE security_orchestration_policy_rule_schedules SET next_run_at = '2024-05-28 00:15:00+00' WHERE id = <rule_schedule_id>;
- Trigger the schedule in the rails console
Security::ScanExecutionPolicies::RuleScheduleWorker.perform_async(project_id, 1, rule_schedule_id)
Go to Build > Pipelines
Verify that a pipeline with 5 jobs was created
8. Go to ‘:9090/graph’ and search for the security_policies_active_builds_scheduled_scans