Skip to content

Support multiple dbs when dropping partitions

What does this MR do and why?

Related to #336898 (closed)

This MR continues the work from !68795 (merged) to make dynamic partition management work with multiple databases. The first MR implemented multi-database functionality for partition creating/detachment, but this MR adds multi-database support to the recently added logic to clean up detached partitions.

Since we don't have an explicit partitioned model to reference as in !68795 (merged), we loop over each existing database and check the detached_partitions table for partitions local to that database.

How to set up and validate locally

In the rails console:

  1. Create a "partition" to drop
      ActiveRecord::Base.connection.create_table 'gitlab_partitions_dynamic.my_test_table'
  2. Create the DetachedPartition record
      Postgresql::DetachedPartition.create!(drop_after: 1.month.ago, table_name: 'my_test_table')
  3. Run the worker
      Database::DropDetachedPartitionsWorker.new.perform 
  4. Verify the table is removed
      ActiveRecord::Base.connection.table_exists?('gitlab_partitions_dynamic.my_test_table') # => false
  5. Verify the DetachedPartition record is removed
      Postgresql::DetachedPartition.where(table_name: 'my_test_table').exists? # => false

In the log/application_json.log, you should see output similar to:

{"severity":"INFO","time":"2021-09-16T19:14:42.369Z","correlation_id":null,"message":"Dropping detached postgres partitions"}
{"severity":"DEBUG","time":"2021-09-16T19:14:42.369Z","correlation_id":null,"message":"Switched database connection","connection_name":"main"}
{"severity":"INFO","time":"2021-09-16T19:14:42.372Z","correlation_id":null,"message":"Checking for previously detached partitions to drop"}
{"severity":"INFO","time":"2021-09-16T19:14:42.383Z","correlation_id":null,"message":"Dropped previously detached partition","partition_name":"my_test_table"}
{"severity":"INFO","time":"2021-09-16T19:14:42.386Z","correlation_id":null,"message":"Finished dropping detached postgres partitions"}

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Patrick Bair

Merge request reports