Skip to content

Save background migration exceptions into sampling artifacts

What does this MR do and why?

Currently we don't display the success / failure of background migrations (in the comment generated by db:gitlabcom-database-testing job).

These changes will ensure we generate sampling files with correct data for later use in https://gitlab.com/gitlab-org/database-team/gitlab-com-database-testing project.

How to set up and validate locally

  1. Create a new Batched Background Migration
bundle exec rails g batched_background_migration failure_migration --table_name=users --column_name=id --feature_category=database

Use the following classes:

# db/post_migrate/xxx_queue_failure_migration.rb
class QueueFailureMigration < Gitlab::Database::Migration[2.1]
  MIGRATION = "FailureMigration"
  DELAY_INTERVAL = 2.minutes
  BATCH_SIZE = 300
  SUB_BATCH_SIZE = 100

  disable_ddl_transaction!

  restrict_gitlab_migration gitlab_schema: :gitlab_main

  def up
    queue_batched_background_migration(
      MIGRATION,
      :users,
      :id,
      job_interval: DELAY_INTERVAL,
      batch_size: BATCH_SIZE,
      sub_batch_size: SUB_BATCH_SIZE
    )
  end

  def down
    delete_batched_background_migration(MIGRATION, :users, :id, [])
  end
end

# lib/gitlab/background_migration/failure_migration.rb
module Gitlab
  module BackgroundMigration
    class FailureMigration < BatchedMigrationJob
      operation_name :update_all
      feature_category :database

      def perform
        each_sub_batch do |_sub_batch|
          random_action.sample.call
        end
      end

      private

      def random_action
        [
          -> () { raise 'Error!' },
          -> () { Logger.new($stdout).info('== MIGRATED ==') }
        ]
      end
    end
  end
end
  1. Add random some users
2500.times do |i|
  begin
    User.create!(
      username: FFaker::Internet.user_name,
      name: FFaker::Name.name,
      email: FFaker::Internet.email,
      confirmed_at: DateTime.now,
      password: SecureRandom.hex.slice(0,16)
    )
  rescue ActiveRecord::RecordInvalid
    print 'Fail'
  end
end
  1. Run migrations. Migration will fail, because we're raising an error in the FailureMigration class
bundle exec rails db:migrate
  1. Run BBM sampling tests
mkdir -p tmp/migration-testing/main/background_migrations
echo "0" > tmp/migration-testing/main/background_migrations/last-batched-background-migration-id.txt
bundle exec rake gitlab:db:migration_testing:sample_batched_background_migrations:main
  1. Check your sampling files
  • Check tmp/migration-testing/main/background_migrations/FailureMigration/batch_<number>/migration-stats.json
  • If the batch failed, it will shown the success: false and the error_message: "Error!". Otherwise, success should be true and error_message: null

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to gitlab-org/database-team/gitlab-com-database-testing#65 (closed)

Edited by Leonardo da Rosa

Merge request reports