Skip to content

Remove existing repository backups when creating a full backup

James Fargher requested to merge clean_repository_backups into master

What does this MR do and why?

Fixes gitaly#4244 (closed)

gitaly-backup was designed to be able to manage its own filesystem, this means that every time you ask it to take a full backup of a repository, it tries to create a new directory to put that full backup into. This makes less sense when we are trying to create a single full backup locally using SKIP=tar. It turns out that the files and DB subtasks already erase existing backups. So repository backups were the odd one out.

This MR makes sure the repository backup directory is removed before taking a full backup.

Note that the test coverage here looks scant, but there are already tests for creating an incremental backup on an untarred backup and tests for restores.

I've raised an issue to discuss a better solution to managing untarred backups #362981

How to set up and validate locally

  1. Create an untarred backup 
    $ bundle exec rake gitlab:backup:create SKIP=tar
...
2022-05-20 10:24:30 +1200 -- Backup 1652999032_2022_05_20_15.0.0-pre is done.
  2. Find a repository to check. Notice the directory that matches the just created backup name. 
    $ tree tmp/backups/repositories/@hashed/19/58/19581e27de7ced00ff1ce50b2047e7a567c76b1cbaebabe5ef03f7c3017bb5b7/
tmp/backups/repositories/@hashed/19/58/19581e27de7ced00ff1ce50b2047e7a567c76b1cbaebabe5ef03f7c3017bb5b7/
├── 1652999032_2022_05_20_15.0.0-pre
│   ├── 001.bundle
│   ├── 001.refs
│   └── LATEST
└── LATEST

1 directory, 4 files
  3. Create another untarred backup without removing the existing one. 
    $ bundle exec rake gitlab:backup:create SKIP=tar
...
2022-05-20 10:29:01 +1200 -- Backup 1652999308_2022_05_20_15.0.0-pre is done.
  4. Check that there's still only a single full backup in the test repository. 
    $ tree tmp/backups/repositories/@hashed/19/58/19581e27de7ced00ff1ce50b2047e7a567c76b1cbaebabe5ef03f7c3017bb5b7/
tmp/backups/repositories/@hashed/19/58/19581e27de7ced00ff1ce50b2047e7a567c76b1cbaebabe5ef03f7c3017bb5b7/
├── 1652999308_2022_05_20_15.0.0-pre
│   ├── 001.bundle
│   ├── 001.refs
│   └── LATEST
└── LATEST

1 directory, 4 files

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by James Fargher

Merge request reports