Provide status of repository shard migrations to the administrator
Problem to solve
An administrator needs to send an API call to GitLab to move a repository from shard a to shard b. This would schedule a background job that actually moves the data. The administrator receives a 200 status code almost right away, which does not indicate anything on the background job, only that the jobs was scheduled. The administrator will now be in the dark of the status of data migration, unless the migration was successful and the migration updated the shard information the admin has no clue what's going on.
Intended users
Proposal
GitLab should provide a good way to get the job status to the user. We could start off by having a POST and GET API endpoint if possible, and allow the administrator to use that. I think ultimately the goal would be an interface somewhat like our GitHub importer
Permissions and Security
Moving data around is for administrators only, and this should not change.
What does success look like, and how can we measure that?
Currently @nnelson is moving data around at gitlab.com, he's got a script that's long polling for the change in repository_storage. If the job failed, the long poll retried to often. If his script is able to detect success as fast as it does now, and failures just as fast, that would be a success.
Links / references
- GitHub importer UI: https://docs.gitlab.com/ee/user/project/import/github.html#selecting-which-repositories-to-import
Availability & Testing
- No risk to availability expected.
- New unit and integration API tests should provide adequate coverage.
- However, a new E2E test is also recommended because we currently only have two E2E tests involving non-default repository storage, both covering more than just the basic functionality, so it would be useful to have a simpler test so that we can more easily diagnose the cause of failures.
- The E2E framework currently polls for a change in the repository storage name to confirm that a requested storage change has taken place. It should be updated to use this new feature instead.