Skip to content

Add repository bundle export to relations export API

George Koltsov requested to merge georgekoltsov/add-repo-bundle-export-to-relations-export-api into master

What does this MR do and why?

This MR adds project repository bundle export to the file of relations that are exported as part of Project Relations Export API (https://docs.gitlab.com/ee/api/project_relations_export.html).

It's going to be used in GitLab Migration (which includes groups and projects when bulk_import_projects feature flag is enabled) feature in order to import project repository from one GitLab instance to another.

Export process is pretty much identical to the one in file-based Project Export's RepoSaver.

The project repository import has already been implemented using fetch_as_mirror Gitaly API. However, fetch_as_mirror does not migrate keep-around git refs, that are needed when importing MRs with source branches that no longer exist (deleted). Such MRs fail to preserve diff notes, since the we fail to fetch diff files from the repository.

Repository bundle presenves keep-around refs, which we need when migrating MR diff notes.

This MR is first part of two in order to fix #358776 (closed) :

  1. [This MR ▶] Add repository export bundle to relations export API
  2. [Future MR, after this one is merged] Update BulkImports::Projects::Pipelines::RepositoryPipeline to use repository bundle instead of fetch_as_mirror API. Keep using fetch_as_mirror for older versions of GitLab.

How to set up and validate locally

  1. Create a project with repository
  2. Start relations export for a project curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/<project_id>/export_relations"
  3. Verify repository bundle export is finished using https://gitlab.example.com/api/v4/projects/<project_id>/export_relations/status endpoint. Status should become 1
  4. Download exported bundle using https://gitlab.example.com/api/v4/projects/<project_id>/export_relations/download?relation=repository_bundle. It should start download of repository_bundle.tar.gz file

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by George Koltsov

Merge request reports