feat(glsh): gitaly repository move
What
- Create a new command
glsh gitalyto allow SREs to work with Gitaly nodes. - Create
glsh gitaly repository moveto move a repository by project id to new storage. The caller doesn't have control over which storage it goes to and depends on the current Gitaly nodes' weights.
Examples
gstg:
$ export GITLAB_ADMIN_PAT=xxx
$ glsh gitaly repository move gstg 7715422
env=gstg project_id=7715422 msg=scheduling repository for a storage move
env=gstg project_id=7715422 src=nfs-file22 dest=nfs-file-cny01 msg=schduled repository for storage move
env=gstg project_id=7715422 src=nfs-file22 dest=nfs-file-cny01 msg=waiting for repository move to finish
env=gstg project_id=7715422 src=nfs-file22 dest=nfs-file-cny01 status=started msg=repository storage move status
env=gstg project_id=7715422 src=nfs-file22 dest=nfs-file-cny01 status=finished msg=repository storage move status
[ gstg ] production> Project.find_by_id(7715422).repository_storage
=> "nfs-file22"
[ gstg ] production> Project.find_by_id(7715422).repository_storage
=> "nfs-file-cny01"
gprd:
$ export GITLAB_ADMIN_PAT=xxx
$ glsh gitaly repository move gprd 10644153
env=gprd project_id=10644153 msg=scheduling repository for a storage move
env=gprd project_id=10644153 src=nfs-file83 dest=nfs-file84 msg=schduled repository for storage move
env=gprd project_id=10644153 src=nfs-file83 dest=nfs-file84 msg=waiting for repository move to finish
env=gprd project_id=10644153 src=nfs-file83 dest=nfs-file84 status=started msg=repository storage move status
env=gprd project_id=10644153 src=nfs-file83 dest=nfs-file84 status=finished msg=repository storage move status
[ gprd ] production> Project.find_by_id(10644153).repository_storage
=> "nfs-file83"
[ gprd ] production> Project.find_by_id(10644153).repository_storage
=> "nfs-file84"
Why
We need an easy and automated way to move a repository from 1 storage to another when a repository is too busy for 1 server and to better balance storage nodes. In the past we've done this by hand sending requests to the API, the idea is now.
We will create a runbook on how to use this after we succesfully migrate some projects in https://gitlab.com/gitlab-com/gl-infra/capacity-planning/-/issues/372
Reference: https://gitlab.com/gitlab-com/gl-infra/capacity-planning/-/issues/372