Draft: Re-enable the v2 prompts endpoint for chat with a frame parser

Andrew Dunnrequested to merge
fix/classic-chat-v2-prompt-response-parser into revert-chat-prompts-v2

What does this MR do and why?

!256499 took the v2 prompts endpoint away from Duo Chat because the chat code could not read what it returns. This restores it behind ai_prompts_v2, with that reader, on top of the revert's branch; it retargets to master when the revert merges.

/v2/prompts/chat streams newline-delimited PromptResponse JSON where /v1/prompts sends bare text, and BufferedIo cuts it on a 16 KiB buffer, so a fragment is a slice, not a frame. Three consumers read it:

  • slash commands answer raw NDJSON at :ok;
  • reader tools raise JSON::ParserError and burn MAX_RETRIES calls;
  • the streaming block broadcasts partial JSON.

Requests::PromptResponseParser buffers fragments, splits on the delimiter and yields each frame's content, so split and doubled frames both survive. An unreadable frame is skipped, logged by length and never by text. #request keys on the endpoint, not the bytes: a v1 answer often starts with a brace. The v1 arm answers with the client's body, and leaves with the flag in !255476.

flowchart TD
  A["Tool executor"] --> B["AiGateway#request"]
  B --> C{"ai_prompts_v2?"}
  C -->|off| D["/v1/prompts/chat<br/>bare text"]
  C -->|on| E["/v2/prompts/chat<br/>NDJSON frames"]
  D --> F["Client#stream"]
  E --> F
  F -->|v1| G["client body"]
  F -->|v2| H["PromptResponseParser<br/>frame content"]
  G --> I["answer"]
  H --> I

No changelog: ai_prompts_v2 is default-disabled (doc/development/feature_flags/_index.md:918).

References

  • !256499, the revert this follows.
  • !255476, the flag removal this unblocks.
  • #552131, the rollout.
  • #466072, the follow-up that reads these frames' token counts.
  • No CODEOWNERS entry matches chain/requests/: route to @gitlab-org/ai-powered/duo-chat and group::ai framework, the flag's owner.

How to set up and validate locally

  1. Feature.enable(:ai_prompts_v2, User.find_by_username('root'))
  2. Summarize issue <url>: three POST /v2/prompts/chat/issue_reader calls and three Json parsing error lines, then M3003, become one and none.
  3. /explain: raw NDJSON at :ok becomes prose.
  4. With the flag off, v1 is unchanged.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist.

🤖 Generated with Claude Code

Merge request reports