How Dependency Proxy via Workhorse works

Injectors

sequenceDiagram
    Client->>+Workhorse: Request
    Workhorse->>+Rails: Propagate the request as-is
    Rails-->>-Workhorse: Respond with a special header that contains instructions for proceeding with the request
    Workhorse-->>Client: Response

An example: Send Git Blob

sequenceDiagram
    Client->>Workhorse: Request for a blob
    Workhorse->>Rails: Propagate the request
    
    Rails->>Workhorse: Respond with a `git-blob:{encoded_data}` header
    Workhorse->>Gitaly: Establish a connection to perform GetBlob RPC
    Gitaly->>Workhorse: Stream the data
    Workhorse->>Client: Stream the data

How GitLab Rails processes the request?

• send_git_blob
• Send a header with a particular information

How Workhorse processes the header?

• Specify a list of injectors
• Iterate over injectors to find a match
• Process a particular request

An example: Send File

sequenceDiagram
    Client->>Workhorse: Request for a blob
    Workhorse->>Rails: Propagate the request
    
    Rails->>Workhorse: Respond with a `send-url:{encoded_data}` header
    Workhorse->>Object Storage: Establish a connection
    Object Storage->>Workhorse: Stream the data
    Workhorse->>Client: Stream the data

---

Pre-authorized requests

sequenceDiagram
    participant Client
    participant Workhorse
    participant Rails
    participant Object Storage

    Client->>+Workhorse: PUT /artifacts/uploads
    Note right of Rails: Append `/authorize` to the original URL and call Rails for an Auth check
    Workhorse->>+Rails: GET /artifacts/uploads/authorize
    Rails-->>-Workhorse: Authorized successfully

    Client->>+Workhorse: Stream the file content
    Workhorse->>+Object Storage: Upload the file
    Object Storage-->>-Workhorse: Success

    Workhorse->>+Rails: Finalize the request
    Note right of Rails: Call the original URL to create a database record
    Rails-->>-Workhorse: Finalized successfully
    Workhorse-->>Client: Uploaded successfully

Docs to read:

• https://docs.gitlab.com/ee/development/uploads/working_with_uploads.html
• https://docs.gitlab.com/ee/development/uploads/index.html#workhorse-assisted-uploads

---

Dependency Proxy

Issue

#11548 (closed)

sequenceDiagram
    Client->>Rails: Request for a file
    alt In Cache
        Rails->>Client: Respond with the file (using send-url injector)
    else Not In Cache
        Rails->>Container Registry: Request to download the file
        Container Registry->>Rails: Respond with the file
        Rails->>Object Storage: Upload file
        Object Storage->>Rails: Uploaded
        Rails->>Client: Respond with the file
    end

sequenceDiagram
    Client->>Workhorse: GET /v2/*group_id/dependency_proxy/containers/*image/blobs/:sha
    Workhorse->>Rails: GET /v2/*group_id/dependency_proxy/containers/*image/blobs/:sha 
    Rails->>Rails: Check DB. Is blob persisted in cache?
    
    alt In Cache
        Rails->>Workhorse: Respond with send-url injector
        Workhorse->>Client: Send the file to the client
    else Not In Cache
        Rails->>Rails: Generate auth token and download URL for the blob in upstream registry
        Rails->>Workhorse: Respond with send-dependency injector
        Workhorse->>Container Registry: Open stream for a blob
        Container Registry->>Workhorse: Stream
        Workhorse->>Rails: GET /v2/*group_id/dependency_proxy/containers/*image/blobs/:sha/authorize
        Rails->>Workhorse: Respond with upload instructions
        Workhorse->>Client: Send the file to the client
        Workhorse->>Object Storage: Save the file
        Workhorse->>Rails: Finalize the upload
    end

Merge request:

• !71890 (merged)