Scope MarkDoneFinalizedMergeRequestTodos filters to the sub-batch ids

What does this merge request do and why?

The MarkDoneFinalizedMergeRequestTodos batched background migration failed twice on GitLab.com with statement timeouts. The 19.3 re-queue with smaller batches also failed, so the batch size was not the cause.

Each cursor sub-batch is a keyset relation with ORDER BY id LIMIT sub_batch_size. The job added its filters and the merge_requests join onto that relation, so the LIMIT counted matching rows, not rows in the sub-batch. Where matches are rare, each sub-batch scanned to the end of the batch. On a Database Lab clone, one sub-batch of a failed range took 95 seconds with the old query and 43 ms with the new query.

The job now filters sub_batch.model.where(id: sub_batch.select(:id)). This is the same id IN (sub_batch.select(:id)) pattern that other batched background migrations use. The 19.3 re-queue migration becomes a no-op, and a new 19.5 migration deletes the failed run and queues it again with larger batch sizes (50_000 / 1_000), which the scoped sub-batches now allow.

References

Screenshots or screen recordings

Before After

Merge request acceptance checklist

Evaluate this merge request against the merge request acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Phil Hughes

Merge request reports

Loading
Loading