Skip to content

Add backup option to filter for specific projects

James Fargher requested to merge backup_projects into master

What does this MR do and why?

The repositories for specific projects can be backed up or restored using the env var REPOSITORIES_PATHS which takes a comma separated list of project paths.

In the future this feature may be expanded to backup all the repositories for groups.

How to set up and validate locally

Backup a specific project

  1. Create a backup specifying a project
    $ bundle exec rake gitlab:backup:create BACKUP=flight REPOSITORIES_PATHS=flightjs/Flight
    2022-05-24 16:03:54 +1200 -- Dumping database ... 
    Dumping PostgreSQL database gitlabhq_development ... [DONE]
    2022-05-24 16:03:56 +1200 -- Dumping database ... done
    2022-05-24 16:03:56 +1200 -- Dumping repositories ... 
    {"command":"create","gl_project_path":"flightjs/Flight","level":"info","msg":"started create","relative_path":"@hashed/e7/f6/e7f6c011776e8db7cd330b54174fd76f7d0216b612387a5ffcfb81e6f0919683.git","storage_name":"default","time":"2022-05-24T04:03:56.744Z"}
    {"command":"create","gl_project_path":"flightjs/Flight","level":"info","msg":"completed create","relative_path":"@hashed/e7/f6/e7f6c011776e8db7cd330b54174fd76f7d0216b612387a5ffcfb81e6f0919683.git","storage_name":"default","time":"2022-05-24T04:03:56.767Z"}
    {"command":"create","gl_project_path":"flightjs/Flight.wiki","level":"info","msg":"started create","relative_path":"@hashed/e7/f6/e7f6c011776e8db7cd330b54174fd76f7d0216b612387a5ffcfb81e6f0919683.wiki.git","storage_name":"default","time":"2022-05-24T04:03:56.802Z"}
    {"command":"create","gl_project_path":"flightjs/Flight","level":"info","msg":"started create","relative_path":"@hashed/e7/f6/e7f6c011776e8db7cd330b54174fd76f7d0216b612387a5ffcfb81e6f0919683.design.git","storage_name":"default","time":"2022-05-24T04:03:56.803Z"}
    {"command":"create","error":"manager: repository empty: repository skipped","gl_project_path":"flightjs/Flight","level":"warning","msg":"skipped create","relative_path":"@hashed/e7/f6/e7f6c011776e8db7cd330b54174fd76f7d0216b612387a5ffcfb81e6f0919683.design.git","storage_name":"default","time":"2022-05-24T04:03:56.804Z"}
    {"command":"create","error":"manager: repository empty: repository skipped","gl_project_path":"flightjs/Flight.wiki","level":"warning","msg":"skipped create","relative_path":"@hashed/e7/f6/e7f6c011776e8db7cd330b54174fd76f7d0216b612387a5ffcfb81e6f0919683.wiki.git","storage_name":"default","time":"2022-05-24T04:03:56.804Z"}
    2022-05-24 16:03:56 +1200 -- Dumping repositories ... done
    ...
    2022-05-24 16:03:57 +1200 -- Backup flight is done.
  2. Check that only this project got added to the backup archive.
    $ tar -tf tmp/backups/flight_gitlab_backup.tar 
    backup_information.yml
    db/database.sql.gz
    repositories/
    repositories/@hashed/
    repositories/@hashed/e7/
    repositories/@hashed/e7/f6/
    repositories/@hashed/e7/f6/e7f6c011776e8db7cd330b54174fd76f7d0216b612387a5ffcfb81e6f0919683/
    repositories/@hashed/e7/f6/e7f6c011776e8db7cd330b54174fd76f7d0216b612387a5ffcfb81e6f0919683/flight/
    repositories/@hashed/e7/f6/e7f6c011776e8db7cd330b54174fd76f7d0216b612387a5ffcfb81e6f0919683/flight/001.refs
    repositories/@hashed/e7/f6/e7f6c011776e8db7cd330b54174fd76f7d0216b612387a5ffcfb81e6f0919683/flight/001.bundle
    repositories/@hashed/e7/f6/e7f6c011776e8db7cd330b54174fd76f7d0216b612387a5ffcfb81e6f0919683/flight/LATEST
    repositories/@hashed/e7/f6/e7f6c011776e8db7cd330b54174fd76f7d0216b612387a5ffcfb81e6f0919683/LATEST
    uploads.tar.gz
    builds.tar.gz
    artifacts.tar.gz
    pages.tar.gz
    lfs.tar.gz
    terraform_state.tar.gz
    packages.tar.gz

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by James Fargher

Merge request reports