Skip to content

Fix partition manager to run against selected dbs

Patrick Bair requested to merge pb-per-database-partition-management into master

What does this MR do and why?

Update partition management to be run against a subset of available databases. This allows us to add single-database rake task variants for creating partitions for main or ci only.

Additionally in places where we hook into single-database rake tasks, like db:migrate:main we can appropriately create partitions for only that database, rather than for all databases. This makes behavior less surprising, as it wouldn't be expected that rake task for main would connect to and run operations on other databases.

To keep this MR manageable, I've made the changes to support the above operations. I'll update the rake tasks to work with the new behavior in the follow up !82717 (merged)

How to set up and validate locally

  1. Setup your application to run with multiple databases https://docs.gitlab.com/ee/development/database/multiple_databases.html#development-setup
  2. Start a rails console connecting to both main and ci
    $ GITLAB_USE_MODEL_LOAD_BALANCING=true rails c
  3. From the console, run partition management with no options
    Gitlab::Database::Partitioning.sync_partitions
  4. Observe in the output that it checks the status for all partitioned tables
    • On main (or main_replica with load balancing)
      • audit_events
      • web_hook_logs
      • loose_foreign_keys_deleted_records
      • batched_background_migration_job_transition_logs
      • incident_management_pending_alert_escalations
      • incident_management_pending_issue_escalations
      • verification_codes
    • On ci (or ci_replica with load balancing)
      • loose_foreign_keys_deleted_records
  5. From the console, run partition management for only the ci database
    Gitlab::Database::Partitioning.sync_partitions(only_on: 'ci')
  6. Observe in the output that it checks the status only for ci tables
    • On ci (or ci_replica with load balancing)
      • loose_foreign_keys_deleted_records

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Patrick Bair

Merge request reports