Skip to content

datastore: Capture and pass `ReplicationEventExistsError`

Karthik Nayak requested to merge 3940-avoid-duplicate-jobs-in-the-replication-queue into master

See the earlier MR and consequent discussion here: !4528 (closed)

migrations: Add unique index replication_queue_constraint

To ensure we don't duplicate replication events in the replication_queue table. Let's create a new unique index 0 which adds a constraint on the following columns

  • state
  • job

This is safer than using a WHERE NOT clause during the insert, as there is no chances of race condition. It is also faster, because now we only hit the index and don't scan all rows of the table.

The migration first deletes any existing duplicates, before adding the new constraint.

With this is in place, we can now capture and propagate this error. In datastore.queue we create a new error type: ReplicationEventExistsError. This error is captured in the datastore.Enqueue function by checking for the replication_queue_constraint unique constraint violation. We propagate this error back.

In praefect.newRequestFinalizer we capture this error, log it and simply move on because this is an expected scenario and nothing to stop execution for.

Part of #3940 (closed)

Edited by Karthik Nayak

Merge request reports