fix(orbit): stream raw response body in 'remote query'

Dmitry Gruzdrequested to merge
fix/orbit-query-llm-response-decode into main

What does this MR do and why?

Fixes a regression on the default code path of glab orbit remote query: every query that uses the default (or explicit) llm response_format fails with

Invalid character '@' looking for beginning of value.

in glab v1.97.0, regardless of query correctness. This is a different defect from the one fixed by !3241 (merged) — that MR addressed @ on the input side; this one addresses @ on the response side.

Symptom

cat > /tmp/q.json <<'JSON'
{"query":{"query_type":"traversal","node":{"id":"p","entity":"Project","filters":{"id":{"op":"eq","value":278964}}},"limit":1}}
JSON
glab orbit remote query /tmp/q.json
# ERROR  Invalid character '@' looking for beginning of value.

The --format raw path happened to work — but only by coincidence: the server happens to return JSON in that case.

Root cause

For response_format=llm, the Orbit API returns GOON/TOON text (Content-Type: text/plain; charset=utf-8) whose first byte is @ (header marker):

$ glab api -X POST orbit/query --header "Content-Type: application/json" --input /tmp/llm.json
@header
query_type:traversal
goon_version:1.0.0
nodes:1
edges:0
@nodes
Project(1):
278964 name=GitLab
@edges

The previous CLI path went through client.Lab().Orbit.Query(...) in the client-go SDK, which uses do[*OrbitQueryResult] → client.Do(req, &OrbitQueryResult{}) → json.NewDecoder(resp.Body).Decode(...). That decoder rejects the leading @ with invalid character '@' looking for beginning of value, which Fang's error renderer sentence-cases to Invalid character '@' ....

!3241 (merged) documented and tightened the input-side path so a stray @ outside a string literal in the user's request body produces a helpful hint pointing at jq and clarifying that @ inside string values is fine. None of that touches the response-side decode, which is why the failure persisted in v1.97.0.

Fix

Bypass the typed OrbitService.Query helper and stream the response body verbatim:

  • client.Lab().NewRequest(POST, "orbit/query", req, ...) builds the request (Content-Type, auth, and User-Agent are still set automatically).
  • client.Lab().Do(httpReq, &bytes.Buffer{}) instructs the SDK to io.Copy the body into the buffer instead of decoding it as JSON. The go-gitlab Client.Do switch-cases on v's type: when v is an io.Writer, the body is copied verbatim (see gitlab.go line 1247-1249).
  • The buffer is written to stdout.

Both formats now round-trip the server's bytes unchanged. orbiterr.Translate continues to map non-2xx responses (401/403/404/429/4xx/5xx) to the existing exit codes — that path runs before the body decode in client.Do.

Why bypass the typed SDK call

The typed *OrbitQueryResult shape only makes sense for response_format=raw (JSON envelope). For llm, the body is not JSON at all, so any client-side decode is wrong by construction. The CLI's job is to forward the server's payload (raw JSON or GOON text) to the user — exactly what the new code does. The MR description on !3241 (merged) already declared the contract:

The body is not preprocessed before JSON parsing: bytes inside JSON string literals (including @) are forwarded verbatim.

This MR extends the same "forward verbatim" guarantee to the response side.

Tests

The mock-based tests in this package previously asserted on the gitlabtesting.MockOrbit.Query interceptor; that mock cannot exercise the JSON-decoding behaviour because it short-circuits the wire-format path entirely. They are rewritten on top of httptest.NewServer, so the test handler observes the exact request the CLI sends and controls the exact response bytes the CLI consumes. This matches the existing convention in internal/commands/search/semantic/semantic_test.go.

New regressions:

  • TestQuery_LLMResponseStreamedVerbatim — sends a multi-line GOON body starting with @header and asserts byte-equality with stdout. Locks in that no JSON decode of the response is attempted.
  • TestQuery_RawResponseStreamedVerbatim — sends a compact raw JSON envelope and asserts byte-equality with stdout. Guards against any future client-side re-marshalling that would reorder keys or change whitespace.

All previously existing scenarios are preserved in the rewritten tests: flag-vs-body precedence (TestQuery_FlagOverridesBodyResponseFormat, TestQuery_BodyFormatHonoredWhenNoFlag), file vs stdin paths, @-in-string-literal preservation (TestQuery_FromFile_WithAtInStrings, TestQuery_Stdin_WithAtInStrings), stray-@ error hint, BOM stripping, and 401 → ExitUnauthenticated mapping.

How to set up and validate locally

Click to expand 
# 1. Default `llm` format now works (was broken before this MR).
cat > /tmp/q.json <<'JSON'
{"query":{"query_type":"traversal","node":{"id":"p","entity":"Project","filters":{"id":{"op":"eq","value":278964}}},"limit":1}}
JSON
./bin/glab orbit remote query /tmp/q.json
# → @header
#   query_type:traversal
#   ...

# 2. --format raw still works unchanged.
./bin/glab orbit remote query --format raw /tmp/q.json
# → {"result":{"format_version":"2.0.0",...},"query_type":"traversal","row_count":1}

# 3. !3241's input-side error hint still fires for stray `@`.
cat > /tmp/bad.json <<'JSON'
{"query": @variable}
JSON
./bin/glab orbit remote query /tmp/bad.json
# →  Query body is not valid JSON: stray '@' outside a string literal
#    at byte 11 (1-indexed)...

# 4. Run the new tests
cd ~/workspace/cli
go test ./internal/commands/orbit/remote/query/...
E2E test results (production gitlab.com, knowledge_graph FF enabled)

glab column is unpatched v1.97.0; patched is this branch built locally.

# Test Command glab v1.97.0 patched Notes
1 Project lookup, default (llm) glab orbit remote query q.json ❌ exit 1, Invalid character '@' ✅ exit 0, GOON body The bug
2 Project lookup, --format raw glab orbit remote query --format raw q.json ✅ exit 0, JSON ✅ exit 0, JSON Already worked
3 Multi-node traversal, default (llm) 2-node MR-in-Project traversal ❌ exit 1, Invalid character '@' ✅ exit 0, GOON body
4 Multi-node traversal, --format raw same as #3 (closed) with --format raw ✅ exit 0, JSON ✅ exit 0, JSON
5 Aggregation, default (llm) count(MergeRequest) where IN_PROJECT(GitLab) ❌ exit 1, Invalid character '@' ✅ exit 0, GOON body
6 Aggregation, --format raw same as #5 (closed) with --format raw ✅ exit 0, JSON mr_count: 230095 ✅ exit 0, same
7 Input has @ in JSON string literal, default username: "@dgruzd", default format ❌ exit 1, Invalid character '@' ✅ exit 0, GOON Result-side bug, not input
8 Input has @ in JSON string literal, raw same as #7 (closed) with --format raw ✅ exit 0, JSON ✅ exit 0, JSON
9 --format llm overrides body response_format=raw --format llm flag wins ❌ exit 1 (llm path broken) ✅ exit 0, GOON
10 Body response_format: llm (no flag) flag absent, body picks llm ❌ exit 1 ✅ exit 0, GOON
11 Body response_format: raw (no flag) flag absent, body picks raw ✅ exit 0, JSON ✅ exit 0, JSON
12 Stray @ outside string literal {"query": @variable} ✅ exit 1, !3241 (merged) hint¹ ✅ exit 1, same Input-side error path
13 HTTP 400 schema error bad neighbors body ✅ exit 1, wrapped error ✅ exit 1, same orbiterr.Translate

¹ The !3241 (merged) input-side hint is only present in builds containing commit 1a815c8 (merged after v1.97.0 was tagged). v1.97.0 itself shows only the bare stdlib message for stray @; the patched build here is rebased on top of !3241 (merged) so the helpful hint is included for completeness.

Bytes-on-the-wire sanity: for the same 2-node multi-node query, llm returns 465 bytes of GOON vs 697 bytes of raw JSON — a 33% size reduction. Streaming the body verbatim preserves that token savings for agents.

Compatibility

No CLI surface changes: same command, same flags, same exit codes. Behaviour change is strictly additive — the previously-broken llm path now returns a response body instead of a stdlib decoder error.

The Long help has been updated to spell out the new contract ("server response is written to stdout verbatim regardless of format — no client-side decoding or re-encoding is performed") and is regenerated via make gen-docs.

/cc @phikai

Merge request reports