Cap Files API content SHA header at 300 MiB

Why

The Repository Files API computes X-Gitlab-Content-Sha256 by streaming the whole blob from Gitaly and hashing it in Rails, under Gitaly's 10s fast timeout. For large files, receiving the blob through grpc-ruby takes longer than that on CPU alone, so the request fails with a 500 GRPC::DeadlineExceeded. !266600 (merged) fixed the memory use but not the timeout.

Omitting the header for files above a size cap turns those failing requests into successful ones. The cap is set below where these timeouts start, with headroom, so it mostly affects requests that would fail anyway.

What

  • Builds on !257771 (closed) (first commit, by @vyaklushin).
  • Behind the files_api_content_sha_size_cap feature flag (gitlab_com_derisk, disabled by default), omits X-Gitlab-Content-Sha256 for blobs over 300 MiB on HEAD file, GET/HEAD blame and GET/HEAD raw. Above the cap, the SHA is never computed or read from the cache.
  • GET /projects/:id/repository/files/:file_path (JSON) still computes the SHA, because it already loads the full content.
  • With the flag off, behaviour is unchanged.
  • Documents the behaviour in doc/api/repository_files.md.

How to validate locally

Works on GDK or caproni. You need a file over 300 MiB in a project, a personal access token with api scope, and a Rails console.

  1. Push a large file with git. The Files and Commits APIs can't create files this size.

    mkdir sha-cap && cd sha-cap && git init -b big
    head -c 440000000 /dev/urandom > big.bin   # 440 MB, random so git can't compress it
    git add big.bin && git commit -m "Add 440 MB test blob"
    git -c http.postBuffer=1048576000 push http://<gitlab-host>/<group>/<project>.git big
  2. Set up the request loop:

    export HOST=http://<gitlab-host> TOKEN=<pat> PROJECT=<group>%2F<project>
    
    check() {
      for p in "" /raw /blame; do
        echo "HEAD big.bin$p"
        curl -sI -H "PRIVATE-TOKEN: $TOKEN" "$HOST/api/v4/projects/$PROJECT/repository/files/big.bin$p?ref=big" \
          | grep -iE '^(HTTP|x-gitlab-content-sha256|x-gitlab-size)'
      done
      echo "GET big.bin/raw"
      curl -s -o /dev/null -D - -H "PRIVATE-TOKEN: $TOKEN" "$HOST/api/v4/projects/$PROJECT/repository/files/big.bin/raw?ref=big" \
        | grep -iE '^(HTTP|x-gitlab-content-sha256|x-gitlab-size)'
    }
  3. In the Rails console, clear any cached SHA and enable the flag:

    project = Project.find_by_full_path('<group>/<project>')
    blob = project.repository.blob_at(project.commit('big').sha, 'big.bin')
    Rails.cache.delete("blob_content_sha256:#{project.full_path}:#{blob.id}")
    Feature.enable(:files_api_content_sha_size_cap, project)

    Run check. Expect 200, X-Gitlab-Size, and no X-Gitlab-Content-Sha256. The HEAD requests return quickly because the blob is not streamed.

  4. Disable the flag with Feature.disable(:files_api_content_sha_size_cap) and run check. Expect X-Gitlab-Content-Sha256 on every response. The first request is slow because it computes the SHA, and the result is cached.

  5. Enable the flag again and run check. Expect the header to stay absent, even though a SHA is now cached.

  6. Optional: confirm the value from step 4 with shasum -a 256 big.bin.

Feature flag changes can take up to a minute to reach a running Puma process.

Results on caproni (440 MB random binary)
Flag SHA cached HEAD file HEAD raw GET raw HEAD blame
on no 200, header absent, 0.49s 200, header absent 200, header absent, full body 200, header absent, 0.16s
off no 200, header present, 7.16s (computes SHA) 200, header present 200, header present 200, header present

The header matched shasum -a 256 of the file. GET /files/:file_path (JSON) still returned content_sha256 above the cap.

Edited by Hunter Stewart

Merge request reports

Loading
Loading