Skip to content

Restore repositories from backup manifest files

James Fargher requested to merge restore_specific_backups into master

What does this MR do and why?

When no backup ID is provided to gitaly-backup the latest backup is used. For non-server-side backups this means the only backup in the tar file. The problem with the latest backup is that using latest is not compatible with manifest files. Since manifest files allow us to properly set the default branch and also to eventually support sha256 repos, here we begin sending the backup ID to gitaly-backup unconditionally.

Now that gitaly-backup sets both the default branch and the hash format it is not possible to tentatively create repositories that did not exist in the backup. There is not enough information to do this. So the tests were adjusted to suit.

How to set up and validate locally

Best way to verify this is to check Migrating to a new server via backup/restore ch... (#416107 - closed). This means setting a default branch that cannot be stored in a bundle file.

  1. Find a project that has a repository in rails console: 
    [1] pry(main)> Project.find(1).repository.exists?
=> true
  2. Set the default branch to a branch that does not exist: 
    [2] pry(main)> Project.find(1).repository.raw.write_ref('HEAD', 'refs/heads/does_not_exist')
=> <Gitaly::WriteRefResponse: >
  3. Check that this worked: 
    [3] pry(main)> Project.find(1).repository.raw.root_ref(head_only: true)
=> "does_not_exist"
  4. Remove any existing backups. This saves any confusing clashes. 
    $ rm -rf $GDK_ROOT/gitlab/tmp/backups/*
  5. Create a backup. Skipping DB so that it is faster and less invasive. 
    $ bundle exec rake gitlab:backup:create SKIP=db
...
2023-10-25 21:49:42 UTC -- Backup 1698270542_2023_10_25_16.6.0-pre is done.
  6. Restore from this backup: 
    $ bundle exec rake gitlab:backup:restore BACKUP=1698270542_2023_10_25_16.6.0-pre
...
2023-10-25 21:53:45 UTC -- Restore task is done.
  7. Check that the repo still has the default branch set correctly (to the non-existent branch): 
    [1] pry(main)> Project.find(1).repository.raw.root_ref(head_only: true)
=> "does_not_exist"

🎉 🎉 🎉 🎉

Note: If these steps did not work for you, double check that $GDK_ROOT/gitaly is checked out to the same revision as $GDK_ROOT/gitlab/GITALY_SERVER_VERSION. In this case, you will have to start from scratch as the manifest files will not have all the needed information.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by James Fargher

Merge request reports