Skip conflicting inserts for finding identifiers
What does this MR do and why?
Makes Security::Ingestion::Tasks::IngestFindingIdentifiers use ON CONFLICT DO NOTHING instead of DO UPDATE.
vulnerability_occurrence_identifiers holds only (occurrence_id, identifier_id). Its unique key covers the whole row, so a conflicting insert has nothing to update — but DO UPDATE rewrote the row anyway, taking an exclusive row lock held until the slice transaction committed.
Measured over one hour on patroni-sec:
| Value | |
|---|---|
| Updates per hour | 2,132,063 |
| Inserts per hour | 199,259 |
| Updates per insert | 10.7 |
| Index entries written per hour by the non-HOT share | 3,049,396 |
Every one of those 2.1M updates is a no-op by construction. That is ~592 wasted row rewrites per second, each one also taking a lock on a table that is under lock contention.
How it is implemented
Rather than special-casing this task, Gitlab::Ingestion::BulkInsertableTask gains an on_conflict setting so any task whose unique key covers every column worth writing can opt in:
self.unique_by = %i[occurrence_id identifier_id].freeze
self.on_conflict = :nothingThe DO NOTHING path now also passes unique_by through to bulk_insert!. Without it Postgres would emit a bare ON CONFLICT DO NOTHING with no conflict target, which skips on any constraint violation rather than the intended key. Existing callers of that path pass no unique_by, so nothing changes for them.
The task sets no self.uses, so nothing consumed RETURNING and no caller depended on the update.
No feature flag: the table has no mutable payload, so there is no behavioural difference beyond not rewriting rows. The rollback is a revert.
References
- Closes #629989 (closed)
- RCA, including the measurements above: #629530
- Parent epic: &23596
Screenshots or screen recordings
No UI change.
How to set up and validate locally
bundle exec rspec ee/spec/lib/gitlab/ingestion/bulk_insertable_task_spec.rb ee/spec/services/security/ingestion/tasks/ingest_finding_identifiers_spec.rbLocally: 7 examples, 0 failures. RuboCop clean. The specs are behavioural rather than SQL-matching — they backdate an existing row's updated_at and assert it is unchanged after ingestion, which fails under DO UPDATE and passes under DO NOTHING.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist.