Finding UUID Migration Phase 2: Backfill new_uuid for existing findings
Blocked
Blocked by: #558135
This BBM requires security_project_tracked_context_id to be populated on existing records, which is the responsibility of #558135. Until that backfill completes, this BBM will only process the small number of records that already have tracked contexts (new findings ingested after the feature was enabled).
What does this MR do and why?
Implements Phase 2 of the Finding UUID migration (#587798).
This creates a batched background migration (BBM) to backfill new_uuid for existing vulnerability_occurrences records that have a tracked context but no new_uuid value yet.
Changes
- Add
BackfillFindingNewUuidBBM class - Add queue migration
QueueBackfillFindingNewUuid - Add comprehensive specs using
BatchedBackgroundMigrationHelpers::V1::TableHelpers
How does it work?
The migration:
- Selects findings where
new_uuid IS NULLANDsecurity_project_tracked_context_id IS NOT NULL - Generates context-aware UUIDs using the same algorithm as Phase 1:
- report_type
- primary_identifier_fingerprint
- location_fingerprint
- project_id
- security_project_tracked_context_id
- Updates records in batches to avoid locking issues
Findings without a tracked context are skipped (they will remain NULL until Phase 3 determines the approach).
Database Analysis
Records to backfill: ~23.7M records have new_uuid IS NULL AND security_project_tracked_context_id IS NOT NULL
Why db-testing showed no updates: BBM testing samples specific ID ranges. The sampled ranges contained records where either new_uuid was already populated (from Phase 1 ingestion) or security_project_tracked_context_id was NULL.
Query Plans
| Query | Plan Link | Index Used | Data Read |
|---|---|---|---|
| SELECT (fetch records) | postgres.ai |
vulnerability_occurrences_pkey, vulnerability_identifiers_pkey
|
~3 MiB per sub-batch |
| UPDATE (bulk update) | postgres.ai | vulnerability_occurrences_pkey |
~700 KiB per sub-batch |
Migration Plan
| Phase | Issue | MR | Status |
|---|---|---|---|
| 1 | #587625 (closed) | !220835 (merged) | Merged |
| 2 | #587798 | This MR | Ready for Review |
| 3 | #587799 | !220869 | Blocked by Phase 2 |
| 4 | #587800 | !220872 | Blocked by Phase 3 |
| 5 | #587801 | !220873 | Blocked by Phase 4 |
MR acceptance checklist
Please evaluate this MR against the MR acceptance checklist.
Closes #587798