Skip to content

Backfill draft on merge_requests via background migration

What does this MR do?

We had initially added a migration to backfill/correct the new draft column for open merge requests as a post-migration in !62627 (merged), but had to roll it back when it caused timeouts on staging. This MR pushes the functionality into a background migration instead.

Database

Up

== 20210609202501 ScheduleBackfillDraftStatusOnMergeRequests: migrating =======
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:merge_requests, :id, {:where=>"draft = false AND state_id = 1 AND ((title)::text ~* '^\\[draft\\]|\\(draft\\)|draft:|draft|\\[WIP\\]|WIP:|WIP'::text)", :name=>"tmp_index_merge_requests_draft_and_status", :algorithm=>:concurrently})
   -> 0.0189s
-- execute("SET statement_timeout TO 0")
   -> 0.0009s
-- add_index(:merge_requests, :id, {:where=>"draft = false AND state_id = 1 AND ((title)::text ~* '^\\[draft\\]|\\(draft\\)|draft:|draft|\\[WIP\\]|WIP:|WIP'::text)", :name=>"tmp_index_merge_requests_draft_and_status", :algorithm=>:concurrently})
   -> 0.0099s
-- execute("RESET ALL")
   -> 0.0007s
-- Scheduled 0 BackfillDraftStatusOnMergeRequests jobs with a maximum of 100 records per batch and an interval of 120 seconds.

The migration is expected to take at least 0 seconds. Expect all jobs to have completed after 2021-06-10 04:50:39 UTC."
== 20210609202501 ScheduleBackfillDraftStatusOnMergeRequests: migrated (0.0828s)

Down

== 20210609202501 ScheduleBackfillDraftStatusOnMergeRequests: reverting =======
-- transaction_open?()
   -> 0.0000s
-- indexes(:merge_requests)
   -> 0.0159s
-- execute("SET statement_timeout TO 0")
   -> 0.0007s
-- remove_index(:merge_requests, {:algorithm=>:concurrently, :name=>"tmp_index_merge_requests_draft_and_status"})
   -> 0.0046s
-- execute("RESET ALL")
   -> 0.0006s
== 20210609202501 ScheduleBackfillDraftStatusOnMergeRequests: reverted (0.0230s)

Temp Index

exec CREATE INDEX merge_request_draft_temp_index ON merge_requests USING btree (id) WHERE state_id = 1 AND draft = FALSE AND ((title)::text ~* '^\\[draft\\]|\\(draft\\)|draft:|draft|\\[WIP\\]|WIP:|WIP'::text);

The query has been executed. Duration: 52.945 min

\di+ merge_request_draft_temp_index
Session: 4317
Command output:
List of relations
 Schema |              Name              | Type  |    Owner     |     Table      |  Size   | Description 
--------+--------------------------------+-------+--------------+----------------+---------+-------------
 public | merge_request_draft_temp_index | index | joe_acroitor | merge_requests | 5784 kB | 
(1 row)
  • With the index:
explain SELECT id FROM "merge_requests" WHERE "merge_requests"."state_id" = 1 AND "merge_requests"."draft" = FALSE AND "merge_requests"."title" ~* '^\\[draft\\]|\\(draft\\)|draft:|draft|\\[WIP\\]|WIP:|WIP'

Index Only Scan using merge_request_draft_temp_index on public.merge_requests  (cost=0.42..1315.74 rows=58377 width=4) (actual time=0.084..146.856 rows=262468 loops=1)
   Heap Fetches: 6684
   Buffers: shared hit=226833
   I/O Timings: read=0.000 write=0.000

Time: 184.845 ms
  - planning: 18.312 ms
  - execution: 166.533 ms
    - I/O read: 0.000 ms
    - I/O write: 0.000 ms

Shared buffers:
  - hits: 226833 (~1.70 GiB) from the buffer pool
  - reads: 0 from the OS file cache, including disk I/O
  - dirtied: 0
  - writes: 0

Related to #330276 (closed)

Edited by Kerri Miller

Merge request reports