Skip to content

Allow backing up or restoring of repositories in a specific storage

James Fargher requested to merge backup_storages into master

What does this MR do and why?

#233296

Adds a new option REPOSITORIES_STORAGES to backup.rake. This accepts a comma separated list of storages, only these storages will be used to create or restore repositories.

How to set up and validate locally

  1. Update gdk.yml to add more storages:
    gitaly:
      storage_count: 2
  2. Rejigger gdk:
    $ gdk reconfigure && gdk restart
  3. Move some repo to the new storage, it will be called gitaly-1.
    [1] pry(main)> move = Project.find(21).repository_storage_moves.build source_storage_name: 'default', destination_storage_name: 'gitaly-1'
    ...
    [2] pry(main)> move.schedule
    ...
    => true
  4. Take a backup of only this new storage:
    $ bundle exec rake gitlab:backup:create REPOSITORIES_STORAGES=gitaly-1 SKIP=tar
    ...
    2022-05-10 14:43:52 +1200 -- Backup 1652150628_2022_05_10_15.0.0-pre is done.
  5. Check that just that one repo was backed up. Restore works effectively the same. Create a full backup, and you can restore a specific repo.
    $ tree tmp/backups/repositories/
    tmp/backups/repositories/
    └── @hashed
        └── 6f
            └── 4b
                └── 6f4b6612125fb3a0daecd2799dfd6c9c299424fd920f9b308110a2c1fbd8f443
                    ├── 1652150628_2022_05_10_15.0.0-pre
                    │   ├── 001.bundle
                    │   ├── 001.refs
                    │   └── LATEST
                    └── LATEST
    
    5 directories, 4 files

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by James Fargher

Merge request reports