Skip to content

Add internal API to retrieve LFS objects

Stan Hu requested to merge sh-lfs-smudge-api into master

This commit is needed to support the inclusion of LFS files inside a project archive download. Gitaly will access this internal API call to download the file via a custom LFS smudge filter (gitaly!2577 (merged)).

To offload the transfer from the Web application server, we send local files via the Rack Sendfile middleware.

For files stored on object storage, we rely on the Workhorse Send-Data header to make Workhorse retrieve the pre-signed URL.

Part of #15079 (closed)

Data flow:

sequenceDiagram
     Client->>+Workhorse: GET /group/project/-/archive/master.zip
     Workhorse->>+Rails: GET /group/project/-/archive/master.zip
 	Rails->>+Workhorse: Gitlab-Workhorse-Send-Data git-archive
     Workhorse->>Gitaly: SendArchiveRequest
     Gitaly->>Git: git archive master
     Git->>Smudge: OID 12345
     Smudge->>+Workhorse: GET /internal/api/v4/lfs?oid=12345&gl_repository=project-1234
     Workhorse->>+Rails: GET /internal/api/v4/lfs?oid=12345&gl_repository=project-1234
     Rails->>+Workhorse: Gitlab-Workhorse-Send-Data send-url
     Workhorse->>Smudge: <LFS data>
     Smudge->>Git: <LFS data>
     Git->>Gitaly: <streamed data>
     Gitaly->>Workhorse: <streamed data>
     Workhorse->>Client: master.zip
Edited by Stan Hu

Merge request reports