Skip to content

Import large gitlab_project exports via rake task

George Koltsov requested to merge large_imports_rake_task into master

What does this MR do?

This MR adds a new rake task to import big project exports that cannot be imported using the UI (due to the size of the project archive). Currently infrastructure team does this using a runbook script which lives outside of main GitLab's codebase, so if anything changes in method signatures / process of the import -- this script can break or be affected by those changes.

We would like to streamline this process a bit, by adding it to the main codebase, adding specs and maintaining it.

As part of this script we also disable direct and background upload, to avoid situations where a huge archive is being uploaded to GCS (while being inside a transaction, which can cause idle transaction timeouts)

More on reasons behind these changes can be found here: https://gitlab.com/gitlab-com/gl-infra/infrastructure/issues/8383

bundle exec rake "gitlab:import_export:import[root, root, testingprojectimport, /path/to/file.tar.gz]"

Screenshots

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

N/A

Security

N/A

Edited by 🤖 GitLab Bot 🤖

Merge request reports