Provide a migration tool for importing existing repositories in to generation tables
We should provide a tool that imports existing repositories that are not receiving writes in to the repositories
tables so they can benefit from feature relying on this information, such as the read distribution.
Options:
-
Walk the primary storage and create a record for each repository it has in the
repositories
and thestorage_repositories
. The automatic reconciler #2717 (closed) would then notice missing records for the secondaries and replicate to them. This should fix any inconsistencies due to dropped replication jobs. This will create a lot of replication jobs potentially but it guarantees the secondaries are brought consistent. However, this will include deleted repositories which did not have the operation properly replicated. -
Gather the information from the GitLab database. This would have the benefit of identifying repositories that should be deleted already.
I think we can go forward with option 1 and later filter out deleted repositories by cross referencing with the GitLab database if needed.
cc @zj-gitlab