Set all NULL `lock_version` values to 0, Part 1
What does this MR do?
In a background migration, sets the value for lock_version
to 0
if it's nil
in the following tables:
- epics
- issues
- merge_requests
There is no real way to roll this back (and neither would we want to, as we'd just be setting a value to nil
!). A major concern is how much time this would take to execute on .com, and if it requires downtime or not
Output of Migration
== 20200128210353 CleanupOptimisticLockingNulls: migrating ====================
-- transaction_open?()
-> 0.0000s
-- index_exists?(:epics, :lock_version, {:where=>"lock_version IS NULL", :algorithm=>:concurrently})
-> 0.0050s
-- execute("SET statement_timeout TO 0")
-> 0.0005s
-- add_index(:epics, :lock_version, {:where=>"lock_version IS NULL", :algorithm=>:concurrently})
-> 0.0126s
-- execute("RESET ALL")
-> 0.0005s
-- transaction_open?()
-> 0.0000s
-- index_exists?(:merge_requests, :lock_version, {:where=>"lock_version IS NULL", :algorithm=>:concurrently})
-> 0.0099s
-- execute("SET statement_timeout TO 0")
-> 0.0005s
-- add_index(:merge_requests, :lock_version, {:where=>"lock_version IS NULL", :algorithm=>:concurrently})
-> 0.0057s
-- execute("RESET ALL")
-> 0.0005s
-- transaction_open?()
-> 0.0000s
-- index_exists?(:issues, :lock_version, {:where=>"lock_version IS NULL", :algorithm=>:concurrently})
-> 0.0094s
-- execute("SET statement_timeout TO 0")
-> 0.0006s
-- add_index(:issues, :lock_version, {:where=>"lock_version IS NULL", :algorithm=>:concurrently})
-> 0.0058s
-- execute("RESET ALL")
-> 0.0005s
== 20200128210353 CleanupOptimisticLockingNulls: migrated (0.2246s) ===========
Time calculations
For each of these I performed EXPLAIN SELECT count(*) table_name WHERE lock_version IS NULL
Batches are 10,000 rows each
Issues table
Aggregate (cost=8126004.15..8126004.16 rows=1 width=8)
-> Seq Scan on issues (cost=0.00..8092817.51 rows=13274657 width=0)
Filter: (lock_version IS NULL)
13274657/10000 = 1,328 batches 1,328 batches * 2 minutes between each = 2,656 minutes = ~45 hours
Merge Requests table
Aggregate (cost=9619905.45..9619905.46 rows=1 width=8)
-> Seq Scan on merge_requests (cost=0.00..9560548.16 rows=23742915 width=0)
Filter: (lock_version IS NULL)
23742915/10000 = 2,375 batches 2,375 batches * 2 minutes between each = 4,749 minutes = ~80 hours
Epics table
Aggregate (cost=5742.34..5742.35 rows=1 width=8)
-> Seq Scan on epics (cost=0.00..5735.46 rows=2750 width=0)
Filter: (lock_version IS NULL)
2750/10000 = less than 1. Single batch to go through everything, estimated time ~2 minutes
Database checklist
-
Conforms to the database guides
When adding migrations:
-
Updated db/schema.rb
-
Added a down
method so the migration can be reverted -
Added the output of the migration(s) to the MR body -
Added tests for the migration in spec/migrations
if necessary (e.g. when migrating data) -
Added rollback procedure. Include either a rollback procedure or description how to rollback changes
When adding or modifying queries to improve performance:
-
Included data that shows the performance improvement, preferably in the form of a benchmark -
Included the output of EXPLAIN (ANALYZE, BUFFERS)
of the relevant queries
When adding foreign keys to existing tables:
-
Included a migration to remove orphaned rows in the source table before adding the foreign key -
Removed any instances of dependent: ...
that may no longer be necessary
When adding tables:
-
Ordered columns based on the Ordering Table Columns guidelines -
Added foreign keys to any columns pointing to data in other tables -
Added indexes for fields that are used in statements such as WHERE
,ORDER BY
,GROUP BY
, andJOIN
s
When removing columns, tables, indexes or other structures:
-
Removed these in a post-deployment migration -
Made sure the application no longer uses (or ignores) these structures
Refs #28560 (closed)