Replicate object pools on repository move
What does this MR do and why?
Contributes to #412407 (closed)
Problem
When projects are moved from shard to shard, we don't move related object pools with them. That prevents repositories from utilizing object pool features.
Solution
Migrate object pools to new shards if they don't exist there. There are several scenarios possible.
Project doesn't have an object pool
- Skip object pool migration and move only project repository
Project has an object pool
- Shard where project is migrated doesn't have an object pool
- Copy the object pool to the new shard
- Create a PoolRepository object in database for the new shard
- Link the project to the new PoolRepository
- Link the project to the new object pool
- Unlink the project from old pool repository
- Shard where project is migrated already has an object pool
- Find PoolRepository object
- Link the project to the PoolRepository
- Link the project to the object pool
- Unlink the project from old pool repository
How to set up and validate locally
- Enable feature flag
Feature.enable(:replicate_object_pool_on_move)
- Configure multiple Gitaly storages. It is possible to do it via GDK: https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/doc/howto/gitaly.md#add-gitaly-nodes
- Create a public project, then fork the project in order to create
pool_repository
- You should see a new record in
pool_repositories
table (PoolRepository.last should have asource_project
link to the original project) - You should see a new folder for the object pool in your GDK (path to pool folder
repositories/@cluster/pools/
) - Call an API request to move a repository to a different storage: https://docs.gitlab.com/ee/api/project_repository_storage_moves.html#schedule-a-repository-storage-move-for-a-project
- You should see a new in
pool_repositories
table (a new PoolRepository should still point to the samesource_project
, but it should have a different shard value) - You should see a new folder for the object pool in your GDK (path to pool folder
repository_storages/praefect-gitaly-1/praefect-internal-1/@cluster/pools/
) - Verify that object pool is replicated as well. Content of related object pools (in
repositories/@cluster/pools/
andrepository_storages/praefect-gitaly-1/praefect-internal-1/@cluster/pools/
) should be the same. - Repeat the migration process for the forked project
- Original
PoolRepository
record should disappear, because all projects were moved to the new shard - Try to migrate project back but in a different order (forked project first, main project next)
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Edited by Vasilii Iakliushin