Skip to content

Create backups using current backup ID

James Fargher requested to merge backup_create_backup_id into master

What does this MR do and why?

#422971 (closed)

Previously the full backup ID was passed to gitaly-backup when backups were being created. This full backup ID was only used when creating a new full backup. A new incremental backup will always use the latest backup ID, queried from the filesystem. So effectively changing this to the current backup ID makes no difference.

We want to start passing the current backup ID so that unique manifest files can be created for each increment.

How to set up and validate locally

  1. Create a full backup.

    $ bundle exec rake gitlab:backup:create
    ...
    2023-09-21 02:46:46 UTC -- Backup 1695264382_2023_09_21_16.4.0-pre is done.
  2. Find a target project to inspect.

    [1] pry(main)> Project.find(10).repository.relative_path
    => "@hashed/4a/44/4a44dc15364204a80fe80e9039455cc1608281820fe2b24f1e5233ade6af1dd5.git"
  3. Push changes to the target repo (anything except a branch delete).

  4. Take an incremental backup.

    $ bundle exec rake gitlab:backup:create INCREMENTAL=yes PREVIOUS_BACKUP=1695264382_2023_09_21_16.4.0-pre
    ...
    2023-09-21 02:48:57 UTC -- Backup 1695264514_2023_09_21_16.4.0-pre is done.
  5. Check that a new increment was added for the target repo. We expect to see files named 002. Strip off the .git from the target repo above and add it to repositories/ for the directory within the tar file.

    $ tar tvf tmp/backups/1695264514_2023_09_21_16.4.0-pre_gitlab_backup.tar repositories/@hashed/4a/44/4a44dc15364204a80fe80e9039455cc1608281820fe2b24f1e5233ade6af1dd5
    drwx------ james/james       0 2023-09-21 14:46 repositories/@hashed/4a/44/4a44dc15364204a80fe80e9039455cc1608281820fe2b24f1e5233ade6af1dd5/
    drwx------ james/james       0 2023-09-21 14:48 repositories/@hashed/4a/44/4a44dc15364204a80fe80e9039455cc1608281820fe2b24f1e5233ade6af1dd5/1695264382_2023_09_21_16.4.0-pre/
    -rw------- james/james     361 2023-09-21 14:48 repositories/@hashed/4a/44/4a44dc15364204a80fe80e9039455cc1608281820fe2b24f1e5233ade6af1dd5/1695264382_2023_09_21_16.4.0-pre/002.refs
    -rw------- james/james     204 2023-09-21 14:46 repositories/@hashed/4a/44/4a44dc15364204a80fe80e9039455cc1608281820fe2b24f1e5233ade6af1dd5/1695264382_2023_09_21_16.4.0-pre/001.refs
    -rw------- james/james  149177 2023-09-21 14:46 repositories/@hashed/4a/44/4a44dc15364204a80fe80e9039455cc1608281820fe2b24f1e5233ade6af1dd5/1695264382_2023_09_21_16.4.0-pre/001.bundle
    -rw------- james/james       3 2023-09-21 14:48 repositories/@hashed/4a/44/4a44dc15364204a80fe80e9039455cc1608281820fe2b24f1e5233ade6af1dd5/1695264382_2023_09_21_16.4.0-pre/LATEST
    -rw------- james/james    1449 2023-09-21 14:48 repositories/@hashed/4a/44/4a44dc15364204a80fe80e9039455cc1608281820fe2b24f1e5233ade6af1dd5/1695264382_2023_09_21_16.4.0-pre/002.bundle
    -rw------- james/james      32 2023-09-21 14:48 repositories/@hashed/4a/44/4a44dc15364204a80fe80e9039455cc1608281820fe2b24f1e5233ade6af1dd5/LATEST

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by James Fargher

Merge request reports