Read artifact archives through ranged requests in get_artifact_file

What does this MR do and why?

Implements the ranged-read option discussed in #629157 (closed) for the get_artifact_file MCP tool. The approach was proposed by Vitali in !254894 (comment 3835046960)

What changed

  • The tool no longer streams the whole artifacts archive into a Tempfile on the Puma node (the use_open_file approach). It now calls Zip::File.open_buffer directly on the IO object the uploader returns.
  • That IO is Gitlab::HttpIO when the archive lives in remote object storage (fetched with 128 KB Range requests), or a plain File when it is on local storage.
  • New class Mcp::Tools::RangedZipIo: a delegator that makes HttpIO acceptable to the rubyzip library. It does two things:
    • Adds an eof probe method, because HttpIO only exposes eof?.
    • Hides HttpIO#path. Without this, Zip::File picks up the path and uses it as the archive filename, and reopening entries later fails with ENOENT.
  • The adapter's code comment documents that it supports sequential reads only. This is a known, intentional limitation.

Spike measurement

Using an in-process HTTP server that honors Range requests, reading a single entry out of a 922 KB test archive fetched 267 KB across 3 requests: two small tail chunks for the central-directory scan, plus one chunk at the target entry's offset. The request count does not grow with archive size — it stays at 3 regardless of how large the archive is, because it is driven by the zip format's structure (central directory at the end, then a seek to the entry), not by the total bytes.

Test coverage

  • New spec using a large archive in object storage, asserting the ranged fetch stays under half the size of the full archive.
  • Unit tests for Mcp::Tools::RangedZipIo.
  • Examples for a failed ranged request and a non-zip archive body, both asserting the download-URL error.
  • All existing behavior specs for this tool pass unchanged.

Limits

  • The 20 MB MAX_ARCHIVE_BYTES cap stays. Ranged reads remove the transfer and disk cost, but Zip::File.open_buffer still loads the whole central directory and builds one Zip::Entry per record, and the MAX_ARCHIVE_ENTRIES check only runs when the metadata file exists. The archive size cap is what bounds that load for an archive without metadata. It can be raised once production numbers are in.
  • MAX_BYTE_OFFSET is tied to MAX_ARCHIVE_BYTES. With HttpIO, skipping into a stored (uncompressed) entry costs one 128 KB Range request per chunk, so the offset ceiling also bounds the request count per call. A stored entry can never be larger than its archive, so the archive cap is the natural ceiling for the offset too. This settles open question 3 in #629157 (closed).
  • A Gitlab::HttpIO::FailedToGetChunkError or Zip::Error during the read returns an error that names the job and its download URL, instead of a generic tool failure.

The companion application setting proposed in !255712 (closed) is not needed with these limits and that MR is closed.

Related to #629157 (closed)

Edited by Tian Gao

Merge request reports

Loading
Loading