Skip to content

Prepare for ci background migration worker

Patrick Bair requested to merge 343047-add-ci-worker-for-bg-migrations into master

What does this MR do and why?

Related to #343047 (closed)

As part of the database decomposition effort, we intend to introduce a new BackgroundMigrationWorker which will process background migrations specifically enqueued to execute against the ci database. The existing BackgroundMigrationWorker will continue to process jobs for the main database.

This MR does some preparation work to add the new worker, by extracting out the worker logic for reuse, and relying on the supplied tracking_database to know where to connect. We also move the tests into a shared example where they can be run against all the workers in generic fashion.

In a future MR, we can add a new BackgroundMigration::CiDatabaseWorker and override the tracking database, and it should work properly.

How to set up and validate locally

The intent of this MR is to keep the current behavior the same, so we can directly exercise the worker to ensure it runs the job:

  1. Setup some test migration job class:
    module Gitlab
      module BackgroundMigration
        class MyTestJob
          def perform(*args)
            puts "args: #{args}"
          end
        end
      end
    end
  2. Run the job inline with the existing worker:
    BackgroundMigrationWorker.new.perform('MyTestJob', [10, 20])
  3. Verify the output looks similar to:
    args: [10, 20]
  4. Schedule the job to run at a later time:
    BackgroundMigrationWorker.perform_in(1.week, 'MyTestJob', [20, 30])
  5. Force the job to run now anyhow:
    Gitlab::BackgroundMigration.steal('MyTestJob')
  6. Verify the output looks similar to:
    args: [20, 30]

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Patrick Bair

Merge request reports