Prepare indexes for bigint column conversions
What does this MR do?
Prepare indexes for asynchronous creation for tables that need to have columns converted from int to bigint. We need to do this for tables which have large indexes on GitLab.com, so deployments can complete in a reasonable time. The indexes should be created as part of the reindexing process that runs every weekend. Async index creation was added in !66478 (merged)
For example, with ci_job_artifacts we have 5 indexes that contain the id or job_id columns, both of which are being converted from int to bigint. The migration in this MR schedules those 5 indexes to be created asynchronously, before the migration to swap the columns is merged. Once the swap migration runs, the add_concurrent_index will be a no-op since the indexes will already exist. On self-managed installations, the indexes will be created as part of the swap migration as normal.
The related MRs for the swaps are: !65601 (merged), !66625 (merged), !66088 (merged)
How to setup and validate locally
Note the migration only inserts a record into postgres_async_indexes for each index, so runtime is not a concern.
rails db:migrate:up VERSION=20210804151444
== 20210804151444 PrepareIndexesForCiJobArtifactBigintConversion: migrating ===
-- index_exists?(:ci_job_artifacts, :id_convert_to_bigint, {:unique=>true, :name=>:index_ci_job_artifact_on_id_convert_to_bigint, :algorithm=>:concurrently})
-> 0.0039s
-- add_index_options(:ci_job_artifacts, :id_convert_to_bigint, {:unique=>true, :name=>:index_ci_job_artifact_on_id_convert_to_bigint, :algorithm=>:concurrently})
-> 0.0000s
-- index_exists?(:ci_job_artifacts, [:project_id, :id_convert_to_bigint], {:where=>"file_type = 18", :name=>:index_ci_job_artifacts_for_terraform_reports_bigint, :algorithm=>:concurrently})
-> 0.0030s
-- add_index_options(:ci_job_artifacts, [:project_id, :id_convert_to_bigint], {:where=>"file_type = 18", :name=>:index_ci_job_artifacts_for_terraform_reports_bigint, :algorithm=>:concurrently})
-> 0.0000s
-- index_exists?(:ci_job_artifacts, :id_convert_to_bigint, {:where=>"file_type = 18", :name=>:index_ci_job_artifacts_id_for_terraform_reports_bigint, :algorithm=>:concurrently})
-> 0.0028s
-- add_index_options(:ci_job_artifacts, :id_convert_to_bigint, {:where=>"file_type = 18", :name=>:index_ci_job_artifacts_id_for_terraform_reports_bigint, :algorithm=>:concurrently})
-> 0.0000s
-- index_exists?(:ci_job_artifacts, [:expire_at, :job_id_convert_to_bigint], {:name=>:index_ci_job_artifacts_on_expire_at_and_job_id_bigint, :algorithm=>:concurrently})
-> 0.0026s
-- add_index_options(:ci_job_artifacts, [:expire_at, :job_id_convert_to_bigint], {:name=>:index_ci_job_artifacts_on_expire_at_and_job_id_bigint, :algorithm=>:concurrently})
-> 0.0000s
-- index_exists?(:ci_job_artifacts, [:job_id_convert_to_bigint, :file_type], {:unique=>true, :name=>:index_ci_job_artifacts_on_job_id_and_file_type_bigint, :algorithm=>:concurrently})
-> 0.0027s
-- add_index_options(:ci_job_artifacts, [:job_id_convert_to_bigint, :file_type], {:unique=>true, :name=>:index_ci_job_artifacts_on_job_id_and_file_type_bigint, :algorithm=>:concurrently})
-> 0.0000s
== 20210804151444 PrepareIndexesForCiJobArtifactBigintConversion: migrated (0.0531s)
rails db:migrate:down VERSION=20210804151444
== 20210804151444 PrepareIndexesForCiJobArtifactBigintConversion: reverting ===
== 20210804151444 PrepareIndexesForCiJobArtifactBigintConversion: reverted (0.0183s)
rails db:migrate:up VERSION=20210804153307
== 20210804153307 PrepareIndexesForTaggingBigintConversion: migrating =========
-- index_exists?(:taggings, :id_convert_to_bigint, {:unique=>true, :name=>:index_taggings_on_id_convert_to_bigint, :algorithm=>:concurrently})
-> 0.0032s
-- add_index_options(:taggings, :id_convert_to_bigint, {:unique=>true, :name=>:index_taggings_on_id_convert_to_bigint, :algorithm=>:concurrently})
-> 0.0000s
-- index_exists?(:taggings, [:taggable_id_convert_to_bigint, :taggable_type], {:name=>:i_taggings_on_taggable_id_convert_to_bigint_and_taggable_type, :algorithm=>:concurrently})
-> 0.0020s
-- add_index_options(:taggings, [:taggable_id_convert_to_bigint, :taggable_type], {:name=>:i_taggings_on_taggable_id_convert_to_bigint_and_taggable_type, :algorithm=>:concurrently})
-> 0.0000s
-- index_exists?(:taggings, [:taggable_id_convert_to_bigint, :taggable_type, :context], {:name=>:i_taggings_on_taggable_bigint_and_taggable_type_and_context, :algorithm=>:concurrently})
-> 0.0019s
-- add_index_options(:taggings, [:taggable_id_convert_to_bigint, :taggable_type, :context], {:name=>:i_taggings_on_taggable_bigint_and_taggable_type_and_context, :algorithm=>:concurrently})
-> 0.0000s
-- index_exists?(:taggings, [:tag_id, :taggable_id_convert_to_bigint, :taggable_type, :context, :tagger_id, :tagger_type], {:unique=>true, :name=>:taggings_idx_tmp, :algorithm=>:concurrently})
-> 0.0018s
-- add_index_options(:taggings, [:tag_id, :taggable_id_convert_to_bigint, :taggable_type, :context, :tagger_id, :tagger_type], {:unique=>true, :name=>:taggings_idx_tmp, :algorithm=>:concurrently})
-> 0.0000s
== 20210804153307 PrepareIndexesForTaggingBigintConversion: migrated (0.0388s)
rails db:migrate:down VERSION=20210804153307
== 20210804153307 PrepareIndexesForTaggingBigintConversion: reverting =========
== 20210804153307 PrepareIndexesForTaggingBigintConversion: reverted (0.0175s)
rails db:migrate:up VERSION=20210804154407
== 20210804154407 PrepareIndexesForCiStageBigintConversion: migrating =========
-- index_exists?(:ci_stages, :id_convert_to_bigint, {:unique=>true, :name=>:index_ci_stages_on_id_convert_to_bigint, :algorithm=>:concurrently})
-> 0.0032s
-- add_index_options(:ci_stages, :id_convert_to_bigint, {:unique=>true, :name=>:index_ci_stages_on_id_convert_to_bigint, :algorithm=>:concurrently})
-> 0.0001s
-- index_exists?(:ci_stages, [:pipeline_id, :id_convert_to_bigint], {:where=>"status in (0, 1, 2, 8, 9, 10)", :name=>:index_ci_stages_on_pipeline_id_and_id_convert_to_bigint, :algorithm=>:concurrently})
-> 0.0022s
-- add_index_options(:ci_stages, [:pipeline_id, :id_convert_to_bigint], {:where=>"status in (0, 1, 2, 8, 9, 10)", :name=>:index_ci_stages_on_pipeline_id_and_id_convert_to_bigint, :algorithm=>:concurrently})
-> 0.0000s
== 20210804154407 PrepareIndexesForCiStageBigintConversion: migrated (0.0262s)
rails db:migrate:down VERSION=20210804154407
== 20210804154407 PrepareIndexesForCiStageBigintConversion: reverting =========
== 20210804154407 PrepareIndexesForCiStageBigintConversion: reverted (0.0140s)
Does this MR meet the acceptance criteria?
Conformity
-
I have included changelog trailers, or none are needed. (Does this MR need a changelog?) -
I have added/updated documentation, or it's not needed. (Is documentation required?) -
I have properly separated EE content from FOSS, or this MR is FOSS only. (Where should EE code go?) -
I have added information for database reviewers in the MR description, or it's not needed. (Does this MR have database related changes?) -
I have self-reviewed this MR per code review guidelines. -
This MR does not harm performance, or I have asked a reviewer to help assess the performance impact. (Merge request performance guidelines) -
I have followed the style guides. -
This change is backwards compatible across updates, or this does not apply.
Availability and Testing
-
I have added/updated tests following the Testing Guide, or it's not needed. (Consider all test levels. See the Test Planning Process.) -
I have tested this MR in all supported browsers, or it's not needed. -
I have informed the Infrastructure department of a default or new setting change per definition of done, or it's not needed.