Let Workhorse serve archived job logs from the trace API

What does this MR do and why?

This MR streams archived job logs from storage instead of buffering them entirely in Rails.

Today, GET /projects/:id/jobs/:job_id/trace reads the whole archived log into memory: for remote storage, hundreds of ranged 128 KiB requests end up held as one Ruby string; for local disk, File#read buffers the whole file too. The job log page avoids this via Workhorse's send_upload, but the API endpoint kept the Rails path because of an old TODO claiming Workhorse couldn't mask runners_token. That's obsolete: secrets are masked when the log is written, not read, so the stored log is already masked.

Behind the new flag ci_job_trace_api_archived_log_via_workhorse (beta, default off, project actor), the endpoint streams instead of buffering. For remote storage, it sends Workhorse a send-url instruction, which Workhorse streams to the client. For local disk, it uses Grape's sendfile, which Rack::Sendfile turns into an X-Sendfile header with an empty body so Workhorse streams the file from disk. HEAD requests are answered from the artifact metadata without fetching the object.

Byte-range requests and live (unarchived) logs still use the old Rails path. The endpoint avoids present_disk_file! and present_carrierwave_file!, since both overwrite the Content-Disposition header this endpoint has always sent (infile; filename="<job id>.log").

Clients see no difference in status, bytes, host, Content-Type, or Content-Disposition. ETag becomes Last-Modified, and Content-Length comes from storage instead of Rails. Storage errors that occur after the archived check are relayed with their own status and body, where the old path returned a JSON 500. One exception: Workhorse forwards Range, If-Range, If-Match, If-None-Match, If-Modified-Since, and If-Unmodified-Since headers to object storage in send-url, and X-Sendfile honors them the same way via Go's http.ServeContent; the current Rails path ignores these headers and always answers 200 with the full body. So with the flag on, a client sending Range now gets 206 with just the requested bytes, and a conditional request can get 304. This only changes behavior for clients that already send those headers.

The flag is beta, not gitlab_com_derisk, because the rollout needs GitLab.com first, then default-on through a full self-managed release before removal; gitlab_com_derisk flags last only two months.

A presigned-URL direct-download redirect is a possible follow-up.

References

• Motivating work item: https://gitlab.com/gitlab-org/gitlab/-/work_items/629424
• Parent epic: https://gitlab.com/groups/gitlab-org/-/work_items/23204
• Rollout issue: https://gitlab.com/gitlab-org/gitlab/-/work_items/629437

Screenshots or screen recordings

Not applicable: API-only change with no UI.

How to set up and validate locally

1. Enable object storage for artifacts on GDK:

   gdk config set object_store.enabled true && gdk reconfigure

2. Run a CI job to completion and wait a couple of minutes for its trace to archive.
3. Note the project ID and the finished job's ID.
4. In a Rails console, enable the flag:

   Feature.enable(:ci_job_trace_api_archived_log_via_workhorse)

5. Fetch the trace with the flag on:

   curl -sv --header "PRIVATE-TOKEN: <token>" \
     "http://gdk.test:3000/api/v4/projects/<project_id>/jobs/<job_id>/trace" \
     -o /tmp/trace_on.log

6. Disable the flag (Feature.disable(:ci_job_trace_api_archived_log_via_workhorse)) and repeat the curl, saving to /tmp/trace_off.log.
7. Compare the two downloads: cmp /tmp/trace_on.log /tmp/trace_off.log. They should be identical.
8. Check the Rails request log for each run: external_http_count should be 1 with the flag on, versus one per 128 KiB chunk with it off.
9. Compare the -v output between the two runs: the flag-on response has Last-Modified where the flag-off response has ETag.
10. With object storage disabled (GDK default), archived logs live under shared/artifacts. With the flag on, download the trace via curl and diff it against a flag-off download; the two should be byte-identical.
11. The streaming mechanism is not visible to the client itself; confirm it only in the Workhorse access log, or by comparing curl -v output with the flag on versus off.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.