Skip conflicting inserts for finding identifiers

What does this MR do and why?

Makes Security::Ingestion::Tasks::IngestFindingIdentifiers use ON CONFLICT DO NOTHING instead of DO UPDATE.

vulnerability_occurrence_identifiers holds only (occurrence_id, identifier_id). Its unique key covers the whole row, so a conflicting insert has nothing to update — but DO UPDATE rewrote the row anyway, taking an exclusive row lock held until the slice transaction committed.

Measured over one hour on patroni-sec:

Value
Updates per hour 2,132,063
Inserts per hour 199,259
Updates per insert 10.7
Index entries written per hour by the non-HOT share 3,049,396

Every one of those 2.1M updates is a no-op by construction. That is ~592 wasted row rewrites per second, each one also taking a lock on a table that is under lock contention.

How it is implemented

Rather than special-casing this task, Gitlab::Ingestion::BulkInsertableTask gains an on_conflict setting so any task whose unique key covers every column worth writing can opt in:

self.unique_by   = %i[occurrence_id identifier_id].freeze
self.on_conflict = :nothing

The DO NOTHING path now also passes unique_by through to bulk_insert!. Without it Postgres would emit a bare ON CONFLICT DO NOTHING with no conflict target, which skips on any constraint violation rather than the intended key. Existing callers of that path pass no unique_by, so nothing changes for them.

The task sets no self.uses, so nothing consumed RETURNING and no caller depended on the update.

No feature flag: the table has no mutable payload, so there is no behavioural difference beyond not rewriting rows. The rollback is a revert.

References

Screenshots or screen recordings

No UI change.

How to set up and validate locally

bundle exec rspec ee/spec/lib/gitlab/ingestion/bulk_insertable_task_spec.rb ee/spec/services/security/ingestion/tasks/ingest_finding_identifiers_spec.rb

Locally: 7 examples, 0 failures. RuboCop clean. The specs are behavioural rather than SQL-matching — they backdate an existing row's updated_at and assert it is unchanged after ingestion, which fails under DO UPDATE and passes under DO NOTHING.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist.

Edited by Bala Kumar

Merge request reports

Loading
Loading