Convert Vulnerability objects to Vulnerabilities::Read object before pushing to ES processing queue

Problem

Currently, Elasticsearch index updates are tracked by monitoring changes in both the Vulnerability and Vulnerabilities::Read models. However, with the introduction of Vulnerabilities Across Contexts, this approach breaks down:

  • The vulnerability_reads table can have multiple entries for the same vulnerability (one per context)
  • The vulnerabilities table has only a single entry per vulnerability
  • Tracking changes based on vulnerability_id alone is insufficient because we can't determine which context-specific read entry needs to be updated

Solution

This issue is preparation work for a parallel Elasticsearch index where the document ID is based on vulnerability_reads.id instead of vulnerability_id. The new index structure will be handled in gitlab-org#21115.

To support this change, convert Vulnerability objects to Vulnerabilities::Read objects before pushing them to the Elasticsearch processing queue. This ensures:

  1. Tracking is done only based on Vulnerabilities::Read model changes
  2. The correct context-specific read entry is identified and updated
  3. Each context gets its own independent Elasticsearch document

Scenarios

Old Index (vulnerability_id as document ID)

When an entry is updated in the vulnerabilities table for which there are multiple entries in the vulnerability_reads table:

  1. All entries from the vulnerability_reads table are enqueued to the ES processing queue
  2. The reference class updates the ES index for all of them
  3. Because the document ID is vulnerability_id, all updates overwrite the same document in the old index

New Index (vulnerability_reads.id as document ID)

When an entry is updated in the vulnerabilities table for which there are multiple entries in the vulnerability_reads table:

  1. All entries from the vulnerability_reads table are enqueued to the ES processing queue
  2. The reference class updates separate documents in the new index
  3. Each document is identified by its unique vulnerability_reads.id, ensuring context-specific updates

Implementation

Update the following components to perform the conversion:

  • Vulnerabilities::BulkEsOperationService - Convert Vulnerability objects to Vulnerabilities::Read when processing bulk operations
  • maintain_elasticsearch_* callback methods in the Vulnerability model - Convert objects in lifecycle callbacks

All changes will be implemented under a feature flag to enable testing first, and then for projects participating in the closed beta.

This change aligns with the Vulnerabilities Across Contexts feature requirements and prepares the codebase for the new ES index structure.

Edited by Rushik Subba