Skip to content
GitLab
Next
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
  • gitlab-workhorse gitlab-workhorse
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Locked Files
  • Issues 0
    • Issues 0
    • List
    • Boards
    • Service Desk
    • Milestones
    • Iterations
    • Requirements
  • Merge requests 0
    • Merge requests 0
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
    • Test Cases
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Code review
    • Insights
    • Issue
    • Repository
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • GitLab.orgGitLab.org
  • gitlab-workhorsegitlab-workhorse
  • Issues
  • #309
Closed (moved) (moved)
Open
Issue created Oct 16, 2020 by Stan Hu@stanhuOwner

Use X-Request-Id for correlation ID from trusted /internal API requests

Right now it's not possible to trace a request all the way through if Gitaly has to make internal API calls back to GitLab Rails. For example, with the new LFS smudge filter added to support gitlab#15079 (closed), we have this sequence of events:

sequenceDiagram
     autonumber
     Client->>+Workhorse: GET /group/project/-/archive/master.zip
     Workhorse->>+Rails: GET /group/project/-/archive/master.zip
 	Rails->>+Workhorse: Gitlab-Workhorse-Send-Data git-archive
     Workhorse->>Gitaly: SendArchiveRequest
     Gitaly->>Git: git archive master
     Git->>Smudge: OID 12345
     Smudge->>+Workhorse: GET /internal/api/v4/lfs?oid=12345&gl_repository=project-1234
     Workhorse->>+Rails: GET /internal/api/v4/lfs?oid=12345&gl_repository=project-1234
     Rails->>+Workhorse: Gitlab-Workhorse-Send-Data send-url
     Workhorse->>Smudge: <LFS data>
     Smudge->>Git: <LFS data>
     Git->>Gitaly: <streamed data>
     Gitaly->>Workhorse: <streamed data>
     Workhorse->>Client: master.zip

At step 7, the LFS smudge filter makes an API call back to Rails. Even if it attempts to preserve the same correlation ID through the request, at step 8 Workhorse will generate a new correlation ID and use that. As a result, the logs for the /api/v4/internal calls all have unique correlation IDs when they should share the same one.

The same issue happens when a Gitaly hook has to make an API call to check /api/v4/internal/allowed, but it's perhaps not as pronounced because there's only 1 or 2 calls.

Some ideas to fix this:

  1. Add a trusted IP or hostname block that allows Workhorse to use X-Request-Id if it is available.
  2. Send a signed token that Workhorse can decode, and if it checks out use the provided correlation ID.

This probably ties into LabKit. What do you think, @andrewn @reprazent?

Assignee
Assign to
Time tracking