Skip to content

Create a new rails generator for Batched Background Migrations

Why?

We still generate Batched Background Migrations through post-deployment generator: bundle exec rails g post_deployment_migration migration_name.

I'm proposing creating a dedicated generator for BBM using a dedicated template.

Example:

bundle exec rails g batched_background_migration my_batched_background_migration
Migration file
class QueueMyBatchedBackgroundMigration < Gitlab::Database::Migration[2.1]
  JOB_CLASS_NAME = 'MyBatchedBackgroundMigrationProcessor'
  TABLE_NAME = :my_table
  BATCH_COLUMN = :id

  def up
    queue_batched_background_migration(JOB_CLASS_NAME, TABLE_NAME, BATCH_COLUMN)
  end

  def down
    delete_batched_background_migration(JOB_CLASS_NAME, TABLE_NAME, BATCH_COLUMN)
  end
end
Migration class
module Gitlab
  module BackgroundMigration
    class MyBatchedBackgroundMigration < ::Gitlab::BackgroundMigration::BatchedMigrationJob
      # operation_name :my_operation
      # scope_to ->(relation) { relation.where(column: "value") }
      # job_arguments :value_a, :value_b
      
      def perform
        # Some generic snippet, like:

        # each_sub_batch do |sub_batch|
        #   Logger.new($stdout).info("Processing #{sub_batch.size} records")
        #
        #   sub_batch.each do |record|
        #     my_record = ::MyRecord.find(record.id)
        #     my_record.name = "Updated name"
        #   end
        # end
      end
    end
  end
end
Migration class specs
# frozen_string_literal: true

require 'spec_helper'

RSpec.describe Gitlab::BackgroundMigration::MyBatchedBackgroundMigration, :migration, schema: 2023...
  # Test
end

Desired Outcome:

  • Create a new generator for batched background migrations
    • Generates the migration file under db/post_migrate
    • Generates the migration class under lib/gitlab/background_migration
    • Generates the migration class spec under spec/lib/gitlab/background_migration
  • Update docs to reflect this change
Edited by Alex Ives