Skip to content

Make migration pause time a configurable value

What does this MR do?

Related to #326840 (closed)

Batched background migration jobs sleep for a small interval between execution of each batch they process. Previously, that pause time had been a fixed value of 0.1 seconds. However, when processing a large number of small batches, that sleep time can dominate the total runtime of the job.

This MR adds a new column to batched_background_migrations and batched_background_migration_jobs, and allows this attribute to be reconfigured on the fly.

Database Migrations

Up

$ bundle exec rails db:migrate:up VERSION=20210413155324
== 20210413155324 AddPauseSecondsToBatchedBackgroundMigrations: migrating =====
-- add_column(:batched_background_migrations, :pause_ms, :integer, {:null=>false, :default=>100})
   -> 0.0043s
== 20210413155324 AddPauseSecondsToBatchedBackgroundMigrations: migrated (0.0043s) 

$ bundle exec rails db:migrate:up VERSION=20210414045322
== 20210414045322 AddPauseSecondsToBatchedBackgroundMigrationJobs: migrating ==
-- add_column(:batched_background_migration_jobs, :pause_ms, :integer, {:null=>false, :default=>100})
   -> 0.0029s
== 20210414045322 AddPauseSecondsToBatchedBackgroundMigrationJobs: migrated (0.0029s) 

Down

$ bundle exec rails db:migrate:down VERSION=20210414045322
== 20210414045322 AddPauseSecondsToBatchedBackgroundMigrationJobs: reverting ==
-- remove_column(:batched_background_migration_jobs, :pause_ms, :integer, {:null=>false, :default=>100})
   -> 0.0024s
== 20210414045322 AddPauseSecondsToBatchedBackgroundMigrationJobs: reverted (0.0058s) 

$ bundle exec rails db:migrate:down VERSION=20210413155324
== 20210413155324 AddPauseSecondsToBatchedBackgroundMigrations: reverting =====
-- remove_column(:batched_background_migrations, :pause_ms, :integer, {:null=>false, :default=>100})
   -> 0.0026s
== 20210413155324 AddPauseSecondsToBatchedBackgroundMigrations: reverted (0.0039s) 

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team
Edited by Krasimir Angelov

Merge request reports