Add single db variant of partitioning rake tasks
What does this MR do and why?
Follow up from !82213 (merged) to update partitioning rake tasks with single-database variants when multiple databases are configured.
With multiple databases configured, partition management can now be run against a single database, like:
$ rails gitlab:db:create_dynamic_partitions:main
Also, for other rake tasks that manage partitions (db:migrate, db:schema:load), this change will only run partition management for the matching database. For example, when running
$ rails db:migrate:ci
it will call rails gitlab:db:create_dynamic_partitions:ci instead of rails gitlab:db:create_dynamic_partitions
How to set up and validate locally
- Setup your development environment with multiple databases as described here: https://docs.gitlab.com/ee/development/database/multiple_databases.html#development-setup
- Verify that the different rake tasks are present:
$ rails -T gitlab:db:create_dynamic_partitions rails gitlab:db:create_dynamic_partitions # Create missing dynamic database partitions rails gitlab:db:create_dynamic_partitions:ci # Create missing dynamic database partitions on the ci database rails gitlab:db:create_dynamic_partitions:main # Create missing dynamic database partitions on the main database - Tail the development log file
$ tail -f log/development.log - Run the ci rake task with:
$ DISABLE_POSTGRES_PARTITION_CREATION_ON_STARTUP=true GITLAB_USE_MODEL_LOAD_BALANCING=true rails gitlab:db:create_dynamic_partitions:ci - Check the log output, to see that partition management only ran for
ci. It should be at the end, after the app is loaded, similar to the following. At this time it will only run forloose_foreign_keysand only fordb_config_name:ci.Creating scope :within_timeframe. Overwriting existing method Iteration.within_timeframe. Creating scope :opened. Overwriting existing method Epic.opened. Creating scope :closed. Overwriting existing method Epic.closed. Creating scope :order_created_desc. Overwriting existing method Packages::Package.order_created_desc. Creating scope :order_name_desc. Overwriting existing method Packages::Package.order_name_desc. Gitlab::Database::PostgresPartition Load (5.1ms) SELECT "postgres_partitions".* FROM "postgres_partitions" WHERE (parent_identifier = concat(current_schema(), '.', 'loose_foreign_keys_deleted_records')) ORDER BY "postgres_partitions"."name" ASC /*application:web,db_config_name:ci,line:/lib/gitlab/database/partitioning/sliding_list_strategy.rb:21:in `map'*/ ↳ lib/gitlab/database/partitioning/sliding_list_strategy.rb:21:in `map' Gitlab::Database::PostgresPartition Load (4.1ms) SELECT "postgres_partitions".* FROM "postgres_partitions" WHERE (parent_identifier = concat(current_schema(), '.', 'loose_foreign_keys_deleted_records')) ORDER BY "postgres_partitions"."name" ASC /*application:web,db_config_name:ci,line:/lib/gitlab/database/partitioning/sliding_list_strategy.rb:21:in `map'*/ ↳ lib/gitlab/database/partitioning/sliding_list_strategy.rb:21:in `map' LooseForeignKeys::DeletedRecord Load (0.8ms) SELECT "loose_foreign_keys_deleted_records"."id", "loose_foreign_keys_deleted_records"."created_at" FROM "loose_foreign_keys_deleted_records" WHERE "loose_foreign_keys_deleted_records"."partition" = 1 ORDER BY "loose_foreign_keys_deleted_records"."id" ASC LIMIT 1 /*application:web,db_config_name:ci,line:/app/models/loose_foreign_keys/deleted_record.rb:20:in `block in <class:DeletedRecord>'*/ ↳ app/models/loose_foreign_keys/deleted_record.rb:20:in `block in <class:DeletedRecord>' Gitlab::Database::PostgresPartition Load (4.1ms) SELECT "postgres_partitions".* FROM "postgres_partitions" WHERE (parent_identifier = concat(current_schema(), '.', 'loose_foreign_keys_deleted_records')) ORDER BY "postgres_partitions"."name" ASC /*application:web,db_config_name:ci,line:/lib/gitlab/database/partitioning/sliding_list_strategy.rb:21:in `map'*/ ↳ lib/gitlab/database/partitioning/sliding_list_strategy.rb:21:in `map' (5.0ms) SELECT columns.column_default AS default_value FROM information_schema.columns columns WHERE columns.column_name = 'partition' AND columns.table_name = 'loose_foreign_keys_deleted_records' /*application:web,db_config_name:ci,line:/lib/gitlab/database/partitioning/sliding_list_strategy.rb:82:in `current_default_value'*/ ↳ lib/gitlab/database/partitioning/sliding_list_strategy.rb:82:in `current_default_value'
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Edited by Patrick Bair