Skip to content

Add partitioning temp indexes to CI tables

Marius Bobin requested to merge 387301-mb-temp-indexes into master

What does this MR do and why?

Related to #387301 (closed)

During gitlab-com/gl-infra/production#8204 (closed) we created records with mixed partition ids that will need to be fixed by updating the partition_id from 101 to 100. But we'll need some indexes to identify the records that need to be fixed on such large tables.

How to set up and validate locally

  1. Feature.enable(:database_async_index_creation)
  2. Feature.enable(:database_reindexing)
  3. GITLAB_SIMULATE_SAAS=1 bin/rails db:migrate
  4. bundle exec rails gitlab:db:reindex
  5. open the PostgreSQL console using the GDK command gdk psql and run the commands
    • \d tmp_index_ci_builds_on_partition_id_and_id
    • \d tmp_index_ci_pipelines_on_partition_id_and_id
    • \d tmp_index_ci_stages_on_partition_id_and_id
$ GITLAB_SIMULATE_SAAS=1 bin/rails db:migrate
main: == 20230117093222 AddTmpIndexToCiBuildsOnPartitionId: migrating ===============
main: -- index_exists?(:ci_builds, [:partition_id, :id], {:name=>:tmp_index_ci_builds_on_partition_id_and_id, :where=>"partition_id = 101", :algorithm=>:concurrently})
main:    -> 0.0103s
main: -- add_index_options(:ci_builds, [:partition_id, :id], {:name=>:tmp_index_ci_builds_on_partition_id_and_id, :where=>"partition_id = 101", :algorithm=>:concurrently})
main:    -> 0.0002s
main: == 20230117093222 AddTmpIndexToCiBuildsOnPartitionId: migrated (0.1705s) ======

main: == 20230117093736 AddTmpIndexToCiPipelinesOnPartitionId: migrating ============
main: -- index_exists?(:ci_pipelines, [:partition_id, :id], {:name=>:tmp_index_ci_pipelines_on_partition_id_and_id, :where=>"partition_id = 101", :algorithm=>:concurrently})
main:    -> 0.0074s
main: -- add_index_options(:ci_pipelines, [:partition_id, :id], {:name=>:tmp_index_ci_pipelines_on_partition_id_and_id, :where=>"partition_id = 101", :algorithm=>:concurrently})
main:    -> 0.0000s
main: == 20230117093736 AddTmpIndexToCiPipelinesOnPartitionId: migrated (0.0112s) ===

main: == 20230117093924 AddTmpIndexToCiStagesOnPartitionId: migrating ===============
main: -- index_exists?(:ci_stages, [:partition_id, :id], {:name=>:tmp_index_ci_stages_on_partition_id_and_id, :where=>"partition_id = 101", :algorithm=>:concurrently})
main:    -> 0.0023s
main: -- add_index_options(:ci_stages, [:partition_id, :id], {:name=>:tmp_index_ci_stages_on_partition_id_and_id, :where=>"partition_id = 101", :algorithm=>:concurrently})
main:    -> 0.0001s
main: == 20230117093924 AddTmpIndexToCiStagesOnPartitionId: migrated (0.0060s) ======

ci: == 20230117093222 AddTmpIndexToCiBuildsOnPartitionId: migrating ===============
ci: -- index_exists?(:ci_builds, [:partition_id, :id], {:name=>:tmp_index_ci_builds_on_partition_id_and_id, :where=>"partition_id = 101", :algorithm=>:concurrently})
ci:    -> 0.0151s
ci: == 20230117093222 AddTmpIndexToCiBuildsOnPartitionId: migrated (0.0244s) ======

ci: == 20230117093736 AddTmpIndexToCiPipelinesOnPartitionId: migrating ============
ci: -- index_exists?(:ci_pipelines, [:partition_id, :id], {:name=>:tmp_index_ci_pipelines_on_partition_id_and_id, :where=>"partition_id = 101", :algorithm=>:concurrently})
ci:    -> 0.0165s
ci: == 20230117093736 AddTmpIndexToCiPipelinesOnPartitionId: migrated (0.0262s) ===

ci: == 20230117093924 AddTmpIndexToCiStagesOnPartitionId: migrating ===============
ci: -- index_exists?(:ci_stages, [:partition_id, :id], {:name=>:tmp_index_ci_stages_on_partition_id_and_id, :where=>"partition_id = 101", :algorithm=>:concurrently})
ci:    -> 0.0037s
ci: -- add_index_options(:ci_stages, [:partition_id, :id], {:name=>:tmp_index_ci_stages_on_partition_id_and_id, :where=>"partition_id = 101", :algorithm=>:concurrently})
ci:    -> 0.0001s
ci: == 20230117093924 AddTmpIndexToCiStagesOnPartitionId: migrated (0.0172s) ======

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Marius Bobin

Merge request reports