Skip to content

Add exclusive lock for UpdateRepositoryStorageWorker

Vasilii Iakliushin requested to merge 429049_fix_storage_move_flow into master

What does this MR do and why?

Contributes to https://gitlab.com/gitlab-org/gitlab/-/issues/429049

Problem

  1. It's possible to run multiple migration workers for the same project/snippet/group simultaneously
  2. The worker can be killed and rescheduled by Sidekiq interrupt signal. It will leave the migration in inconsistent state.

Solution

Use an exclusive lock in storage migration workers. The exclusive lease key includes a project/snippet/group id to prevent simultaneous updates. The key value is a Sidekiq worker jid to track the owner of the update.

This setup should handle following situations:

  1. Worker tries to migrate a repository under existing migration (result: job is marked as failed)
  2. Worker started a migration but was interrupted and rescheduled. (result: job is marked as failed, lock is released)

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Merge request reports