Skip to content

Provide a migration tool for importing existing repositories in to generation tables

We should provide a tool that imports existing repositories that are not receiving writes in to the repositories tables so they can benefit from feature relying on this information, such as the read distribution.

Options:

  1. Walk the primary storage and create a record for each repository it has in the repositories and the storage_repositories. The automatic reconciler #2717 (closed) would then notice missing records for the secondaries and replicate to them. This should fix any inconsistencies due to dropped replication jobs. This will create a lot of replication jobs potentially but it guarantees the secondaries are brought consistent. However, this will include deleted repositories which did not have the operation properly replicated.

  2. Gather the information from the GitLab database. This would have the benefit of identifying repositories that should be deleted already.

I think we can go forward with option 1 and later filter out deleted repositories by cross referencing with the GitLab database if needed.

cc @zj-gitlab

Edited by Sami Hiltunen
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information