Skip to content

gitlab-backup restore leaves the contents of the backup archive in /var/opt/gitlab/backups

Summary

When running gitlab-backup restore BACKUP=... the tar.gz file is extracted to /var/opt/gitlab/backups

This comprises various files and directories:

artifacts.tar.gz
backup_information.yml
builds.tar.gz
lfs.tar.gz
pages.tar.gz
uploads.tar.gz
repositories/
db/

There's also a tmp/ directory that's sometimes created.

if the code did work, tmp/ is not deleted

There's a number of interactive prompts in the process, where the user is given the opportunity to abandon the restore, such as if Puma or Sidekiq are still running.

If this opportunity is taken, the files are left behind.

IF the restore completes successfully, the files are left behind.

severity4 - it's messy, and uses disk space.

Steps to reproduce

  • perform a restore.
  • cd /var/opt/gitlab/backups && ls artifacts.tar.gz backup_information.yml builds.tar.gz lfs.tar.gz pages.tar.gz uploads.tar.gz repositories db

Example Project

What is the current bug behavior?

The restore process leaves behind the contents of the backup archive/

What is the expected correct behavior?

The contents is deleted.

Relevant logs and/or screenshots

# gitlab-backup restore BACKUP=1624385880_2021_06_22_13.12.2-ee
Unpacking backup ... done
Be sure to stop Puma, Sidekiq, and any other process that
connects to the database before proceeding. For Omnibus
installs, see the following link for more information:
https://docs.gitlab.com/ee/raketasks/backup_restore.html#restore-for-omnibus-gitlab-installations

Before restoring the database, we will remove all existing
tables to avoid future upgrade problems. Be aware that if you have
custom tables in the GitLab database these tables and all data will be
removed.

Do you want to continue (yes/no)? no
Quitting...

checking the directory:

# ls
1624385317_2021_06_22_13.11.3-ee_gitlab_backup.tar
1624385880_2021_06_22_13.12.2-ee_gitlab_backup.tar
artifacts.tar.gz
backup_information.yml
builds.tar.gz
db
lfs.tar.gz
pages.tar.gz
repositories
uploads.tar.gz

Output of checks

Results of GitLab environment info

13.12.2

Possible fixes

There's two references to backup.cleanup

      if cleanup_required
        backup.cleanup
      end

cleanup_required contains the state of backup.unpack

The property/state being tested here doesn't actually seem to be right.

modified as follows:

      if cleanup_required
        puts "performing cleanup\n"
        backup.cleanup
      else
        puts "skipping cleanup\n"
      end
Do you want to continue (yes/no)? yes

skipping cleanup
Warning: Your gitlab.rb and gitlab-secrets.json files contain sensitive data

Modified to run backup.cleanup unconditionally:

Deleting tmp directories ... done
done
done
done
done
done
done
done
Warning: Your gitlab.rb and gitlab-secrets.json files contain sensitive data

But

# find tmp
tmp
tmp/uploads.1624458905
tmp/builds.1624458905
tmp/artifacts.1624458905
tmp/pages.1624458905
tmp/lfs.1624458905
tmp/uploads.1624459197
tmp/builds.1624459197
tmp/artifacts.1624459197
tmp/pages.1624459197
tmp/lfs.1624459197

In respect of ask_to_continue, this doesn't appear to return to the backup rake task so getting it to handle the cleanup as part of raise Gitlab::TaskAbortedByUserError may be impossible

Edited by Ben Prescott (ex-GitLab)