How Dependency Proxy via Workhorse works
Injectors
sequenceDiagram
Client->>+Workhorse: Request
Workhorse->>+Rails: Propagate the request as-is
Rails-->>-Workhorse: Respond with a special header that contains instructions for proceeding with the request
Workhorse-->>Client: Response
An example: Send Git Blob
sequenceDiagram
Client->>Workhorse: Request for a blob
Workhorse->>Rails: Propagate the request
Rails->>Workhorse: Respond with a `git-blob:{encoded_data}` header
Workhorse->>Gitaly: Establish a connection to perform GetBlob RPC
Gitaly->>Workhorse: Stream the data
Workhorse->>Client: Stream the data
How GitLab Rails processes the request?
How Workhorse processes the header?
An example: Send File
sequenceDiagram
Client->>Workhorse: Request for a blob
Workhorse->>Rails: Propagate the request
Rails->>Workhorse: Respond with a `send-url:{encoded_data}` header
Workhorse->>Object Storage: Establish a connection
Object Storage->>Workhorse: Stream the data
Workhorse->>Client: Stream the data
Pre-authorized requests
sequenceDiagram
participant Client
participant Workhorse
participant Rails
participant Object Storage
Client->>+Workhorse: PUT /artifacts/uploads
Note right of Rails: Append `/authorize` to the original URL and call Rails for an Auth check
Workhorse->>+Rails: GET /artifacts/uploads/authorize
Rails-->>-Workhorse: Authorized successfully
Client->>+Workhorse: Stream the file content
Workhorse->>+Object Storage: Upload the file
Object Storage-->>-Workhorse: Success
Workhorse->>+Rails: Finalize the request
Note right of Rails: Call the original URL to create a database record
Rails-->>-Workhorse: Finalized successfully
Workhorse-->>Client: Uploaded successfully
Docs to read:
- https://docs.gitlab.com/ee/development/uploads/working_with_uploads.html
- https://docs.gitlab.com/ee/development/uploads/index.html#workhorse-assisted-uploads
Dependency Proxy
Issue
sequenceDiagram
Client->>Rails: Request for a file
alt In Cache
Rails->>Client: Respond with the file (using send-url injector)
else Not In Cache
Rails->>Container Registry: Request to download the file
Container Registry->>Rails: Respond with the file
Rails->>Object Storage: Upload file
Object Storage->>Rails: Uploaded
Rails->>Client: Respond with the file
end
sequenceDiagram
Client->>Workhorse: GET /v2/*group_id/dependency_proxy/containers/*image/blobs/:sha
Workhorse->>Rails: GET /v2/*group_id/dependency_proxy/containers/*image/blobs/:sha
Rails->>Rails: Check DB. Is blob persisted in cache?
alt In Cache
Rails->>Workhorse: Respond with send-url injector
Workhorse->>Client: Send the file to the client
else Not In Cache
Rails->>Rails: Generate auth token and download URL for the blob in upstream registry
Rails->>Workhorse: Respond with send-dependency injector
Workhorse->>Container Registry: Open stream for a blob
Container Registry->>Workhorse: Stream
Workhorse->>Rails: GET /v2/*group_id/dependency_proxy/containers/*image/blobs/:sha/authorize
Rails->>Workhorse: Respond with upload instructions
Workhorse->>Client: Send the file to the client
Workhorse->>Object Storage: Save the file
Workhorse->>Rails: Finalize the upload
end
Merge request:
Edited by Igor Drozdov