<!--IssueSummary start--> <details> <summary> Everyone can contribute. [Help move this issue forward](https://handbook.gitlab.com/handbook/marketing/developer-relations/contributor-success/community-contributors-workflows/#contributor-links) while earning points, leveling up and collecting rewards. </summary> - [Close this issue](https://contributors.gitlab.com/manage-issue?action=close&projectId=278964&issueIid=379223) </details> <!--IssueSummary end--> As discussed in POC https://gitlab.com/gitlab-org/gitlab/-/issues/376261 GitLab Migration should be able to read individual files of the exported `.zip` relations directly from Object Storage instead of downloading the entire zip file. This allows us to decrease disk usage. If a zip file has a collection of large files (e.g. lfs objects) - we can import them one by one instead of downloading all of the at once. ## Proposed solutions 1. Use `gitlab-zip-cat` to read individual files from `.zip` stored in Object Storage. This solution has a few unknowns: - Is `gitlab-zip-cat` workhorse binary available under the same location in all types of gitlab distributions? - Using `gitlab-zip-cat` binary requires making a system call from ruby which is not ideal since it increases malicious user attack surface 2. Implement [`gitlab-zip-cat` functionality](https://gitlab.com/gitlab-org/gitlab/blob/1057e91a9374fc9714a05606ef5250ae946ee7db/workhorse/cmd/gitlab-zip-cat/main.go) in ruby, which uses httpio, seeks specified file and decompresses the contents on the fly All the zip work needs to be done before this. Wait for the batches before the zip.