Skip to content

Resolve "Support multiple databases for batched background migrations"

Diogo Frazão requested to merge 351585-support-multiple-databases-for-batched-background-migrations-3 into master

What does this MR do and why?

Update rake task gitlab:background_migrations:status and gitlab:background_migrations:finalize to support multiple databases.

If only a single database is configured, you should only see the singular rake task gitlab:background_migrations:status/ gitlab:background_migrations:finalize.

If multiple databases are configured, you should see:

rake -T gitlab:background_migrations:finalize
rake gitlab:background_migrations:finalize                                                            # Synchronously finish executing a batched background migration
rake gitlab:background_migrations:finalize:ci[job_class_name,table_name,column_name,job_arguments]    # Gitlab | DB | Synchronously finish executing a batched background migration on ci database
rake gitlab:background_migrations:finalize:main[job_class_name,table_name,column_name,job_arguments]  # Gitlab | DB | Synchronously finish executing a batched background migration on main database
rake -T gitlab:background_migrations:status
rake gitlab:background_migrations:status       # Display the status of batched background migrations
rake gitlab:background_migrations:status:ci    # Gitlab | DB | Display the status of batched background migrations on ci database
rake gitlab:background_migrations:status:main  # Gitlab | DB | Display the status of batched background migrations on main database

When you are running Gitlab with one database:

rake -T gitlab:background_migrations:finalize
rake gitlab:background_migrations:finalize[job_class_name,table_name,column_name,job_arguments]  # Synchronously finish executing a batched background migration
rake -T gitlab:background_migrations:status
rake gitlab:background_migrations:status  # Display the status of batched background migrations

Screenshots or screen recordings

These are strongly recommended to assist reviewers and reduce the time to merge your change.

How to set up and validate locally

Note: Multiple database option needs to be enabled.

  1. Open rails console and run
a = Gitlab::Database::BackgroundMigration::BatchedMigration.new
a.max_value = 40
a.batch_size = 100
a.sub_batch_size = 20
a.interval = 20
a.job_class_name = "example"
a.table_name = "examples"
a.column_name = "ex"

a.save
  1. Run GITLAB_USE_MODEL_LOAD_BALANCING=true rake gitlab:background_migrations:status:ci - you should see 0 records
  2. Run GITLAB_USE_MODEL_LOAD_BALANCING=true rake gitlab:background_migrations:status - you should see 1 record
  3. Run GITLAB_USE_MODEL_LOAD_BALANCING=true rake gitlab:background_migrations:status:main - you should see 1 record

Numbered steps to set up and validate the change are strongly suggested.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #351585 (closed)

Edited by Diogo Frazão

Merge request reports