Import large gitlab_project exports via rake task

Merged George Koltsov requested to merge large_imports_rake_task into master

What does this MR do?

This MR adds a new rake task to import big project exports that cannot be imported using the UI (due to the size of the project archive). Currently infrastructure team does this using a runbook script which lives outside of main GitLab's codebase, so if anything changes in method signatures / process of the import -- this script can break or be affected by those changes.

We would like to streamline this process a bit, by adding it to the main codebase, adding specs and maintaining it.

As part of this script we also disable direct and background upload, to avoid situations where a huge archive is being uploaded to GCS (while being inside a transaction, which can cause idle transaction timeouts)

More on reasons behind these changes can be found here:

bundle exec rake "gitlab:import_export:import[root, root, testingprojectimport, /path/to/file.tar.gz]"


Does this MR meet the acceptance criteria?


Availability and Testing




Edited by George Koltsov