Skip to content

Fix bugs when splitting batched migration jobs

Heinrich Lee Yu requested to merge 332613-fix-bugs-with-split-jobs into master

What does this MR do?

  1. Fetch last migration job by max_value.

    When we split failed jobs, the job with the highest ID will no longer be the one with the highest max_value. These caused new migration jobs to be created for succeeding IDs even though we already have jobs for those.

  2. Make sure jobs are picked up after being split

    The migration runner only picks up existing jobs that are failed with attempts less than the max, or jobs that are stuck in pending or running for an hour.

    We keep the split jobs in the failed state so that they can be picked up immediately.

Migration output

== 20210709085759 IndexBatchedMigrationJobsByMaxValue: migrating ==============
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:batched_background_migration_jobs, [:batched_background_migration_id, :max_value], {:name=>"index_migration_jobs_on_migration_id_and_max_value", :algorithm=>:concurrently})
   -> 0.0047s
-- add_index(:batched_background_migration_jobs, [:batched_background_migration_id, :max_value], {:name=>"index_migration_jobs_on_migration_id_and_max_value", :algorithm=>:concurrently})
   -> 0.0095s
== 20210709085759 IndexBatchedMigrationJobsByMaxValue: migrated (0.0155s) =====
== 20210709085759 IndexBatchedMigrationJobsByMaxValue: reverting ==============
-- transaction_open?()
   -> 0.0000s
-- indexes(:batched_background_migration_jobs)
   -> 0.0043s
-- remove_index(:batched_background_migration_jobs, {:algorithm=>:concurrently, :name=>"index_migration_jobs_on_migration_id_and_max_value"})
   -> 0.0036s
== 20210709085759 IndexBatchedMigrationJobsByMaxValue: reverted (0.0093s) =====

Screenshots or Screencasts (strongly suggested)

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

Does this MR contain changes to processing or storing of credentials or tokens, authorization and authentication methods or other items described in the security review guidelines? If not, then delete this Security section.

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team

Related to #332613 (closed)

Edited by Heinrich Lee Yu

Merge request reports