Skip to content

Skip creating initial partitions for shared database

Thong Kuah requested to merge partition_initialize_no_shared into master

What does this MR do and why?

Skip creating initial partitions for shared database

If the database connection is shared, then the partition will be, or has already been created on that database connection. E.g. if main_clusterwide is shared with main, then creating the partition on main alone is sufficient. No need to perform it on both main_clusterwide and `main.

We need this fix because starting in !120484 (comment 1409579118), rails hangs on boot

Screenshots or screen recordings

Screenshot_2023-05-31_at_1.36.49_PM

How to set up and validate locally

  1. Apply this patch:
diff --git a/lib/gitlab/database/partitioning.rb b/lib/gitlab/database/partitioning.rb
index 3bfcc7162eb6..c5d509d226e2 100644
--- a/lib/gitlab/database/partitioning.rb
+++ b/lib/gitlab/database/partitioning.rb
@@ -41,10 +41,12 @@ def sync_partitions(models_to_sync = registered_for_sync, only_on: nil)
 
               model_connection_name = model.connection_db_config.name
               Gitlab::Database::EachDatabase.each_database_connection(include_shared: false) do |connection, connection_name|
+                puts ["each_database_connection", model.name, connection_name].inspect
                 if connection_name != model_connection_name
                   PartitionManager.new(model, connection: connection).sync_partitions
                 end
               end
+              puts ["end each_database_connection", model.name].inspect
             end
           end
 
diff --git a/lib/gitlab/database/partitioning/sliding_list_strategy.rb b/lib/gitlab/database/partitioning/sliding_list_strategy.rb
index 5bb34a86d43d..98773321ebfc 100644
--- a/lib/gitlab/database/partitioning/sliding_list_strategy.rb
+++ b/lib/gitlab/database/partitioning/sliding_list_strategy.rb
@@ -19,6 +19,7 @@ def initialize(model, partitioning_key, next_partition_if:, detach_partition_if:
 
         def current_partitions
           Gitlab::Database::PostgresPartition.for_parent_table(table_name).map do |partition|
+            puts [table_name, partition.name, partition.condition].inspect
             SingleNumericListPartition.from_sql(table_name, partition.name, partition.condition)
           end.sort
         end
@@ -84,6 +85,7 @@ def validate_and_fix
 
           if old_default_value != expected_default_value
             with_lock_retries do
+              puts "#{model.connection.pool.db_config.name} #{model.connection.load_balancer.configuration.instance_variable_get(:@model).name}: LOCK TABLE #{model.table_name} IN ACCESS EXCLUSIVE MODE"
               model.connection.execute("LOCK TABLE #{model.table_name} IN ACCESS EXCLUSIVE MODE")
 
               old_default_value = current_default_value
@@ -108,6 +110,7 @@ def validate_and_fix
                 new_value: expected_default_value
               )
             end
+            puts "#{model.connection.pool.db_config.name} #{model.connection.load_balancer.configuration.instance_variable_get(:@model).name}: Finished lock for #{model.table_name}"
           end
         end
  1. Configure main_clusterwide database with database_tasks: false. The easiest way is to configure GDK with cells.enabled: true
  2. Run time be rails runner 'puts User.find(1).name'. On master, you can see output relating to creating partitions for main_clusterwide, main and ci. On this branch, we will only see output for main and ci.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Thong Kuah

Merge request reports