Skip to content

Ensure runner taggings are copied from taggings

Marius Bobinrequested to merge
524402-ensure-taggings-exist into master

What does this MR do and why?

Follow-up to !184341 (merged) to ensure that people that have already upgraded to 17.8 before the fix was rolled-out can safely update to 17.9 without data loss.

Why isn't this a background migration?

Because the runners table is changed in 17.10, we can't ensure that the table over which we batch over will be available for the duration of the execution of the background migration. Also we need to have the records created immediately to minimize the system disruption.

References

Screenshots or screen recordings

Manually tested this with a thin clone by using:

marius@rocket-sled ~/W/g/gitlab (524402-ensure-taggings-exist)> pgai connect ci
ℹ info    Data state at: 2025-03-28 08:44:58 UTC
psql (14.17 (Homebrew), server 16.8 (Debian 16.8-1.pgdg110+1))
Type "help" for help.
gitlabhq_dblab=# truncate ci_runner_taggings;
TRUNCATE TABLE
Time: 832.565 ms
gitlabhq_dblab=#
\q

marius@rocket-sled ~/W/g/gitlab (524402-ensure-taggings-exist)> DISABLE_SCHEMA_DUMP=1 DISABLE_POSTGRES_PARTITION_CREATION_ON_STARTUP=1 RAILS_ENV=test pgai use -o ci bin/rails db:migrate:ci
ci: == [advisory_lock_connection] object_id: 121580, pg_backend_pid: 130
ci: == 20250114135814 EnsureRunnerTaggingsExist: migrating ========================
ci: == 20250114135814 EnsureRunnerTaggingsExist: migrated (2485.3954s) ============

The config yielded about 3506 insert batches:

cat log/test.log | grep "INSERT INTO ci_runner_taggings(tag_id," | wc -l
    3506

The total time that we would need to execute this migration for a system that has as much data as .com would be at most 20 minutes:

# 2 queries per batch
# 200ms overhead per connection time to dblab
[14] pry(main)> (2485.3954 / 1.minute) - (3506 * 2 * 200 / 1.second.in_milliseconds / 1.minute )
=> 18.423256666666667

Queries

How to set up and validate locally

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Marius Bobin

Merge request reports