Sync security_finding_enrichments on policy create/update
What does this MR do and why?
When a security approval policy with KEV/EPSS filter rules is created or updated, existing vulnerabilities on the default branch lack security_finding_enrichments records — those are only created during pipeline runs via StoreFindingsService. This means the policy cannot evaluate pre-existing vulnerabilities until a new pipeline runs.
This MR enqueues a background worker when a policy is synced to a project, so that Security::FindingEnrichment records are upserted for all active, default-branch vulnerabilities with CVE identifiers — ensuring enrichment data is available before the first MR evaluation for pre-existing vulnerabilities.
See discussion in &16311 (comment 2999757659) and issue #591177.
This MR is based on the work in:
- !224853 (merged) (Planned to be merged first, the we will rebase this MR)
Database queries
Batch traversal over default-branch vulnerabilities
Batches over the vulnerabilities table using the existing partial index index_vulnerabilities_project_id_and_id_on_default_branch ((project_id, id) WHERE present_on_default_branch IS TRUE)
SELECT "vulnerabilities".*
FROM "vulnerabilities"
WHERE "vulnerabilities"."project_id" = 278964
AND (present_on_default_branch IS TRUE)
AND "vulnerabilities"."id" >= 20641602
ORDER BY "vulnerabilities"."id" ASC
LIMIT 100
Explain: https://console.postgres.ai/gitlab/gitlab-production-sec/sessions/49455/commands/147454
Loading findings with CVE identifiers
SELECT "vulnerability_occurrences".*
FROM "vulnerability_occurrences"
WHERE "vulnerability_occurrences"."id" IN (
SELECT "vulnerabilities"."finding_id"
FROM "vulnerabilities"
WHERE "vulnerabilities"."project_id" = 278964
AND (present_on_default_branch IS TRUE)
AND "vulnerabilities"."id" >= 20641602
ORDER BY "vulnerabilities"."id" ASC
LIMIT 100
)
Explain: https://console.postgres.ai/gitlab/gitlab-production-sec/sessions/49455/commands/147455
CVE identifiers are then eager-loaded.
How to set up and validate locally
- Enable the feature flags:
Feature.enable(:security_policies_kev_filter) Feature.disable(:associate_security_findings_enrichment_records) Feature.enable(:sync_finding_enrichments_on_policy_sync) - Create a project with the following
.gitlab-ci.yml
# See https://docs.gitlab.com/ee/ci/yaml/ for all available options
image: busybox:latest
include:
- template: "Jobs/Dependency-Scanning.gitlab-ci.yml"
- Create an MR with the following file and merge the MR:
# requirements.txt
PyYAML==3.12
- Create an approval policy with an EPSS score or KEV filter rule
---
approval_policy:
- name: Security Scan - Preexisting
description: ''
enabled: true
enforcement_type: enforce
rules:
- type: scan_finding
scanners:
- dependency_scanning
vulnerabilities_allowed: 0
severity_levels: []
vulnerability_states:
- detected
- confirmed
- dismissed
- resolved
branch_type: protected
vulnerability_attributes:
known_exploited: false
epss_score:
operator: greater_than
value: 0.1
actions:
- type: require_approval
approvals_required: 1
role_approvers:
- developer
- maintainer
- owner
- type: send_bot_message
enabled: true
approval_settings:
block_branch_modification: false
prevent_pushing_and_force_pushing: false
prevent_approval_by_author: false
prevent_approval_by_commit_author: false
remove_approvals_with_new_commit: false
require_password_to_approve: false
fallback_behavior:
fail: closed
-
Create another MR by updating the
Readme.mdfile. -
Verify that the MR is blocked by the security policy
Security Scan - Preexisting
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist.