Dual write vulnerability data to both ES indices

What does this MR do and why?

Implements dual-write support for vulnerability data to both Elasticsearch indices: the existing vulnerabilities index and the new vulnerability_reads index.

This is part of the Vulnerabilities Across Contexts (VAC) initiative. The new vulnerability_reads index uses vulnerability_reads.id as primary key (instead of vulnerability_id), which is needed to support multiple context-specific read entries per vulnerability.

Changes

  • Vulnerabilities::Read#elastic_reference — When the vulnerability_read_es_dual_write feature flag is enabled and the create_vulnerability_reads_index ES migration has completed, returns an array of two serialized references (one for each index) instead of a single reference.
  • Search::Elastic::Reference.serialize — Handles Array return values from elastic_reference (pass-through).
  • Elastic::ProcessBookkeepingService#track! — Flattens the serialized items array after mapping, so multi-ref arrays are enqueued as individual items.
  • Feature flag vulnerability_read_es_dual_write — Ops flag (default off) to gate the dual-write behavior.

How it works

Vulnerabilities::Read record
  → elastic_reference
    → [Vulnerability ref string, Vulnerabilities::Read ref string]
  → ProcessBookkeepingService.track!
    → serialize + flatten
    → both refs enqueued independently in Redis
  → ProcessBookkeepingService#execute
    → each ref deserialized and indexed to its respective ES index

Dependencies

This MR is chained on:

Once those merge to master, this MR should be retargeted to master.

Feature flag

Name: vulnerability_read_es_dual_write

Type: wip

The flag is additionally gated on the create_vulnerability_reads_index ES migration having completed, to prevent writes to a non-existent index.

Closes #592362

Edited by Rushik Subba

Merge request reports

Loading