Index vulnerability due dates in Elasticsearch

What does this MR do and why?

First of four stacked MRs for the Vulnerability Report Due date filter. This one adds a due_date field to the vulnerability Elasticsearch indexes. Nothing queries it yet.

New Vulnerability Report filters are ES-only. Due dates live in vulnerability_finding_due_dates (one row per finding), not on vulnerability_reads, so the value has to be looked up at index time:

  • due_date: { type: 'date' } is added to the type mapping, with a mapping migration for each index: vulnerabilities and the dual-written vulnerability_reads.
  • A DueDate preloader for each index reads the finding's due date in one query per batch. It is registered in both EnhancedProxy classes.
  • due_date is added to FIELDS_WITH_MIGRATIONS and PRELOADED_FIELDS, with a new schema version for each index (26_08 and 26_06). The field is only written once its mapping migration has finished.

This mirrors remediated_at (b7ecff6b) for the mapping and latest_flag (e1ce4d8d) for the preloaders.

Stack (each MR targets the one before it):

  1. This MR: ES field and preloaders
  2. !266732 (merged): reindex when a due date changes, plus the backfill
  3. !266733 (merged): ES query and GraphQL arguments, behind vulnerability_due_date_filter
  4. !266734: frontend filter token
Notes for reviewers
  • 20261025120000_add_ascp_security_context_* mapping migrations are already on master, but their write path isn't. If that lands first, it may take schema version 26_08, and this MR's version needs to move up.
  • All specs pass locally against Elasticsearch 9.4.4, including the two :elastic mapping-migration specs.

References

Screenshots or screen recordings

Not applicable: no UI change. Verified in GDK against Elasticsearch 9.4.4: the vulnerabilities index mapping has due_date: { type: 'date' }, and after indexing, the six gitlab-org/gitlab vulnerabilities with a due date have it in their documents.

How to set up and validate locally

Requires Elasticsearch in GDK with advanced vulnerability management.

  1. Run the ES migrations, then check the mapping:

    curl -s localhost:9200/gitlab-development-vulnerabilities/_mapping | jq '.[].mappings.properties.due_date'
  2. Set a due date on a vulnerability (vulnerability details page, or the bulkSetVulnerabilityFindingsDueDates mutation).

  3. Reindex with bundle exec rake gitlab:elastic:index_vulnerabilities, then check that the document has due_date.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

  • Mapping-only migrations, plus one extra batched query per indexing batch.

    Both DueDate preloaders run this query once per indexing batch, with the batch's finding IDs:

    SELECT "vulnerability_finding_due_dates".*
    FROM "vulnerability_finding_due_dates"
    WHERE "vulnerability_finding_due_dates"."vulnerability_occurrence_id" IN (1, 2, 3, ...)

    It uses the unique index index_vulnerability_finding_due_dates_on_occurrence_id, so it returns at most one row per finding.

🤖 Generated with Claude Code

Edited by AJ Biton

Merge request reports

Loading
Loading