Sync security_finding_enrichments on policy create/update

What does this MR do and why?

When a security approval policy with KEV/EPSS filter rules is created or updated, existing vulnerabilities on the default branch lack security_finding_enrichments records — those are only created during pipeline runs via StoreFindingsService. This means the policy cannot evaluate pre-existing vulnerabilities until a new pipeline runs.

This MR enqueues a background worker when a policy is synced to a project, so that Security::FindingEnrichment records are upserted for all active, default-branch vulnerabilities with CVE identifiers — ensuring enrichment data is available before the first MR evaluation for pre-existing vulnerabilities.

See discussion in &16311 (comment 2999757659) and issue #591177.

This MR is based on the work in:

Database queries

Batch traversal over default-branch vulnerabilities

Batches over the vulnerabilities table using the existing partial index index_vulnerabilities_project_id_and_id_on_default_branch ((project_id, id) WHERE present_on_default_branch IS TRUE)

SELECT "vulnerabilities".*
FROM "vulnerabilities"
WHERE "vulnerabilities"."project_id" = 278964
  AND (present_on_default_branch IS TRUE)
  AND "vulnerabilities"."id" >= 20641602
ORDER BY "vulnerabilities"."id" ASC
LIMIT 100

Explain: https://console.postgres.ai/gitlab/gitlab-production-sec/sessions/49455/commands/147454

Loading findings with CVE identifiers

SELECT "vulnerability_occurrences".*
FROM "vulnerability_occurrences"
WHERE "vulnerability_occurrences"."id" IN (
  SELECT "vulnerabilities"."finding_id"
  FROM "vulnerabilities"
  WHERE "vulnerabilities"."project_id" = 278964
    AND (present_on_default_branch IS TRUE)
    AND "vulnerabilities"."id" >= 20641602
  ORDER BY "vulnerabilities"."id" ASC
  LIMIT 100
)

Explain: https://console.postgres.ai/gitlab/gitlab-production-sec/sessions/49455/commands/147455

CVE identifiers are then eager-loaded.

How to set up and validate locally

  1. Enable the feature flags:
    Feature.enable(:security_policies_kev_filter)
    Feature.disable(:associate_security_findings_enrichment_records)
    Feature.enable(:sync_finding_enrichments_on_policy_sync)
  2. Create a project with the following .gitlab-ci.yml
# See https://docs.gitlab.com/ee/ci/yaml/ for all available options
image: busybox:latest
include:
  - template: "Jobs/Dependency-Scanning.gitlab-ci.yml"
  1. Create an MR with the following file and merge the MR:
# requirements.txt

PyYAML==3.12
  1. Create an approval policy with an EPSS score or KEV filter rule
---
approval_policy:
- name: Security Scan - Preexisting
  description: ''
  enabled: true
  enforcement_type: enforce
  rules:
  - type: scan_finding
    scanners:
    - dependency_scanning
    vulnerabilities_allowed: 0
    severity_levels: []
    vulnerability_states:
    - detected
    - confirmed
    - dismissed
    - resolved
    branch_type: protected
    vulnerability_attributes:
      known_exploited: false
      epss_score:
        operator: greater_than
        value: 0.1
  actions:
  - type: require_approval
    approvals_required: 1
    role_approvers:
    - developer
    - maintainer
    - owner
  - type: send_bot_message
    enabled: true
  approval_settings:
    block_branch_modification: false
    prevent_pushing_and_force_pushing: false
    prevent_approval_by_author: false
    prevent_approval_by_commit_author: false
    remove_approvals_with_new_commit: false
    require_password_to_approve: false
  fallback_behavior:
    fail: closed
  1. Create another MR by updating the Readme.md file.

  2. Verify that the MR is blocked by the security policy Security Scan - Preexisting

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist.

Edited by Imam Hossain

Merge request reports

Loading