Skip to content
Snippets Groups Projects

Add backup option to skip specific projects

All threads resolved!

What does this MR do and why?

#18287 (closed)

This MR adds support for SKIP_REPOSITORIES_PATHS environment variable which takes comma separated list of projects or groups and allows to exclude specific projects or groups from repositories backup.

How to set up and validate locally

  1. Create a backup excluding project or group
    bundle exec rake gitlab:backup:create BACKUP=test SKIP_REPOSITORIES_PATHS=giltab-org
  2. Check that excluded projects/groups were not added to backup
    tar -tf tmp/backups/test_gitlab_backup.tar 

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Achilleas Pipinellis removed review request for @axil

    removed review request for @axil

  • Javiera Tapia requested review from @jtapiab

    requested review from @jtapiab

  • Javiera Tapia
  • Thank you @nE0sIghT for this contribution :tada:. I left a few suggestions and comments, please take a look when you have time, they are non-blocking but I highly encourage you to check them if possible.

    For the backend maintainer who is going to review this. The behavior of this contribution works as expected. The results in my local as follow:

    Test case

    • Group: toolbox
    • Project: gitlab-smoke-tests
    Project.find_by_full_path("toolbox/gitlab-smoke-tests").disk_path
    # => "@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b"
    

    Without skipping projects

    Then:

    bundle exec rake gitlab:backup:create BACKUP=test
    tar -tf tmp/backups/test_gitlab_backup.tar | grep "@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b"

    Results:

    repositories/@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b/
    repositories/@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b/test/
    repositories/@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b/LATEST
    repositories/@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b/test/001.bundle
    repositories/@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b/test/LATEST
    repositories/@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b/test/001.refs

    The project with disk_path "@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b" is included in the backup.

    Skipping projects

    bundle exec rake gitlab:backup:create BACKUP=test SKIP_REPOSITORIES_PATHS=toolbox/gitlab-smoke-tests
    tar -tf tmp/backups/test_gitlab_backup.tar | grep "@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b"

    No results are given for this project.

    Tests missing

    • Try a project with snippets, snippets should be excluded too.
    Edited by Javiera Tapia
  • Javiera Tapia approved this merge request

    approved this merge request

  • Javiera Tapia requested review from @ibaum

    requested review from @ibaum

  • added workflowin review label and removed workflowin dev label

  • Javiera Tapia added 1 deleted label

    added 1 deleted label

  • Javiera Tapia removed review request for @jtapiab

    removed review request for @jtapiab

  • Ian Baum requested review from @dbalexandre and removed review request for @ibaum

    requested review from @dbalexandre and removed review request for @ibaum

  • removed review request for @dbalexandre

  • Hi @dbalexandre :wave:

    We noticed this MR is marked as workflowready for review but no reviewer is assigned. workflowin dev has automatically been applied to this MR based on the likelihood the review is finished. If additional reviews are still required, please assign a reviewer and reapply workflowready for review.

    @nE0sIghT you may also request a review by commenting @gitlab-bot ready. You can also assign reviewers directly using @gitlab-bot ready @user1 @user2 if you know the relevant reviewer(s), such as those who were involved in a related issue.

    This message was generated automatically. You're welcome to improve it.

  • added workflowin dev label and removed workflowin review label

  • Author Contributor

    Thanks to all for review! I will handle all threads soon.

  • Yuri Konotopov added 1 commit

    added 1 commit

    • 5fa67b14 - Add backup option to skip specific projects

    Compare with previous version

  • Yuri Konotopov added 1 commit

    added 1 commit

    • df438207 - Add backup option to skip specific projects

    Compare with previous version

  • requested review from @dbalexandre

  • @dbalexandre, this Community contribution is ready for review.

    • Do you have capacity and domain expertise to review this? We are mindful of your time, so if you are not able to take this on, please re-assign to one or more other reviewers.
    • Add the workflowin dev label if the merge request needs action from the author.

    This message was generated automatically. You're welcome to improve it.

  • Douglas Barbosa Alexandre approved this merge request

    approved this merge request

  • resolved all threads

  • Douglas Barbosa Alexandre enabled an automatic merge when the pipeline for 52f02e9f succeeds

    enabled an automatic merge when the pipeline for 52f02e9f succeeds

  • Thanks, @nE0sIghT! This LGTM, MWPS set :tada: :rocket:

  • Contributor
    3 Warnings
    :warning: df438207: The commit body should not contain more than 72 characters per line. For more information, take a look at our Commit message guidelines.
    :warning: This merge request does not refer to an existing milestone.
    :warning: Please add a merge request subtype to this merge request.
    2 Messages
    :book: CHANGELOG missing:

    If you want to create a changelog entry for GitLab FOSS, add the Changelog trailer to the commit message you want to add to the changelog.

    If you want to create a changelog entry for GitLab EE, also add the EE: true trailer to your commit message.

    If this merge request doesn't need a CHANGELOG entry, feel free to ignore this message.

    :book: This merge request adds or changes documentation files. A review from the Technical Writing team before you merge is recommended. Reviews can happen after you merge.

    Documentation review

    The following files require a review from a technical writer:

    The review does not need to block merging this merge request. See the:

    Reviewer roulette

    Changes that require review have been detected!

    Please refer to the table below for assigning reviewers and maintainers suggested by Danger in the specified category:

    Category Reviewer Maintainer
    backend Eugie Limpin current availability (@eugielimpin) (UTC+8) Thong Kuah current availability (@tkuah) (UTC+12)

    To spread load more evenly across eligible reviewers, Danger has picked a candidate for each review slot, based on their timezone. Feel free to override these selections if you think someone else would be better-suited or use the GitLab Review Workload Dashboard to find other available reviewers.

    To read more on how to use the reviewer roulette, please take a look at the Engineering workflow and code review guidelines. Please consider assigning a reviewer or maintainer who is a domain expert in the area of the merge request.

    Once you've decided who will review this merge request, assign them as a reviewer! Danger does not automatically notify them for you.

    If needed, you can retry the :repeat: danger-review job that generated this comment.

    Generated by :no_entry_sign: Danger

  • Contributor

    Allure report

    allure-report-publisher generated test report!

    e2e-test-on-gdk: :exclamation: test report for df438207

    expand test summary
    +-----------------------------------------------------------------------+
    |                            suites summary                             |
    +------------------+--------+--------+---------+-------+-------+--------+
    |                  | passed | failed | skipped | flaky | total | result |
    +------------------+--------+--------+---------+-------+-------+--------+
    | Data Stores      | 2      | 0      | 0       | 1     | 2     | ❗     |
    | Create           | 8      | 0      | 1       | 0     | 9     | ✅     |
    | Govern           | 2      | 0      | 0       | 0     | 2     | ✅     |
    | Monitor          | 4      | 0      | 0       | 0     | 4     | ✅     |
    | Plan             | 4      | 0      | 0       | 0     | 4     | ✅     |
    | Framework sanity | 0      | 0      | 1       | 0     | 1     | ➖     |
    | Manage           | 1      | 0      | 0       | 0     | 1     | ✅     |
    +------------------+--------+--------+---------+-------+-------+--------+
    | Total            | 21     | 0      | 2       | 1     | 23    | ❗     |
    +------------------+--------+--------+---------+-------+-------+--------+
  • @nE0sIghT, how was your code review experience with this merge request? Please tell us how we can continue to iterate and improve:

    1. React with a :thumbsup: or a :thumbsdown: on this comment to describe your experience.
    2. Create a new comment starting with @gitlab-bot feedback below, and leave any additional feedback you have for us in the comment.

    Interested in learning more tips and tricks to solve your next challenge faster? Subscribe to the GitLab Community Newsletter for contributor-focused content and opportunities to level up.

    Thanks for your help! :heart:

    This message was generated automatically. You're welcome to improve it.

  • mentioned in commit 82227b9b

  • 🤖 GitLab Bot 🤖 changed milestone to %16.1

    changed milestone to %16.1

  • added workflowstaging label and removed workflowcanary label

  • mentioned in issue #18287 (closed)

  • Katrin Leinweber mentioned in merge request !140634 (merged)

    mentioned in merge request !140634 (merged)

  • Please register or sign in to reply
    Loading