Support multiple databases for batched background migrations
Batched background migrations need to be updated to support multiple databases, or we will not be able to use them for new migrations leading up to or after the release of database decomposition. Existing migrations that have completed on GitLab.com will be fine, since there are no plans to rollout decomposition to self-managed customers.
We are already updating the redis-based background migrations to work with multiple databases, so this issue is not required to support database decomposition. We will have to support multiple databases before we can consider batched migrations ready for general use, so it should be done in an early iteration of future work.
As a high level to implement this, we have previously discussed:
-
Convert batched migration models to be SharedModel
-
Add a sidekiq cron specifically for the ci
database, which is a separate processing queue frommain
-
Update rake tasks to execute migrations to accept a database argument -
Wrap all entry points to migration execution with SharedModel.using_connection
with the proper connection -
Update generic migration job CopyColumnUsingBackgroundMigrationJob
to use the correct connection -
Update generic batching strategy PrimaryKeyBatchingStrategy
to use the correct connection
Edited by Diogo Frazão