Cap Files API content SHA header at 300 MiB
Why
The Repository Files API computes X-Gitlab-Content-Sha256 by streaming the whole blob from Gitaly and hashing it in Rails, under Gitaly's 10s fast timeout. For large files, receiving the blob through grpc-ruby takes longer than that on CPU alone, so the request fails with a 500 GRPC::DeadlineExceeded. !266600 (merged) fixed the memory use but not the timeout.
Omitting the header for files above a size cap turns those failing requests into successful ones. The cap is set below where these timeouts start, with headroom, so it mostly affects requests that would fail anyway.
What
- Builds on !257771 (closed) (first commit, by @vyaklushin).
- Behind the
files_api_content_sha_size_capfeature flag (gitlab_com_derisk, disabled by default), omitsX-Gitlab-Content-Sha256for blobs over 300 MiB onHEADfile,GET/HEADblame andGET/HEADraw. Above the cap, the SHA is never computed or read from the cache. GET /projects/:id/repository/files/:file_path(JSON) still computes the SHA, because it already loads the full content.- With the flag off, behaviour is unchanged.
- Documents the behaviour in
doc/api/repository_files.md.
Related
- Issue: #604992
- Rollout: #639449
- Previous MR: !257771 (closed)
- Streaming SHA change: !266600 (merged)
How to validate locally
Works on GDK or caproni. You need a file over 300 MiB in a project, a personal access token with api scope, and a Rails console.
-
Push a large file with git. The Files and Commits APIs can't create files this size.
mkdir sha-cap && cd sha-cap && git init -b big head -c 440000000 /dev/urandom > big.bin # 440 MB, random so git can't compress it git add big.bin && git commit -m "Add 440 MB test blob" git -c http.postBuffer=1048576000 push http://<gitlab-host>/<group>/<project>.git big -
Set up the request loop:
export HOST=http://<gitlab-host> TOKEN=<pat> PROJECT=<group>%2F<project> check() { for p in "" /raw /blame; do echo "HEAD big.bin$p" curl -sI -H "PRIVATE-TOKEN: $TOKEN" "$HOST/api/v4/projects/$PROJECT/repository/files/big.bin$p?ref=big" \ | grep -iE '^(HTTP|x-gitlab-content-sha256|x-gitlab-size)' done echo "GET big.bin/raw" curl -s -o /dev/null -D - -H "PRIVATE-TOKEN: $TOKEN" "$HOST/api/v4/projects/$PROJECT/repository/files/big.bin/raw?ref=big" \ | grep -iE '^(HTTP|x-gitlab-content-sha256|x-gitlab-size)' } -
In the Rails console, clear any cached SHA and enable the flag:
project = Project.find_by_full_path('<group>/<project>') blob = project.repository.blob_at(project.commit('big').sha, 'big.bin') Rails.cache.delete("blob_content_sha256:#{project.full_path}:#{blob.id}") Feature.enable(:files_api_content_sha_size_cap, project)Run
check. Expect200,X-Gitlab-Size, and noX-Gitlab-Content-Sha256. The HEAD requests return quickly because the blob is not streamed. -
Disable the flag with
Feature.disable(:files_api_content_sha_size_cap)and runcheck. ExpectX-Gitlab-Content-Sha256on every response. The first request is slow because it computes the SHA, and the result is cached. -
Enable the flag again and run
check. Expect the header to stay absent, even though a SHA is now cached. -
Optional: confirm the value from step 4 with
shasum -a 256 big.bin.
Feature flag changes can take up to a minute to reach a running Puma process.
Results on caproni (440 MB random binary)
| Flag | SHA cached | HEAD file | HEAD raw | GET raw | HEAD blame |
|---|---|---|---|---|---|
| on | no | 200, header absent, 0.49s | 200, header absent | 200, header absent, full body | 200, header absent, 0.16s |
| off | no | 200, header present, 7.16s (computes SHA) | 200, header present | 200, header present | 200, header present |
The header matched shasum -a 256 of the file. GET /files/:file_path (JSON) still returned content_sha256 above the cap.