Fix v4 code suggestion requests sent to the AI Gateway

Mike Wronskirequested to merge
598413-v4-completion-code-context into master

What does this MR do and why?

This MR fixes three problems in what Rails sends to IDE clients and to the AI Gateway /v4/code/suggestions endpoint. That endpoint handles code completions behind the code_completion_v4_endpoint flag, and code generations always.

  1. Since completions moved to v4 (!239271 (merged), rolled out on GitLab.com in July 2026), the IDE's context[] array (open tabs) reached the gateway only as a top-level key that v4 ignores. Context was silently dropped from every completion proxied through Rails. Each context item now becomes a code_context prompt component, placed after the code_editor_completion component. The gateway already consumes these components.
  2. v4 requests no longer include the raw client body (current_file, telemetry, choices_count, and the client-sent model_provider/model_name) next to prompt_components. This duplication doubled the file content in every request and could push a large-context request over the 600 kB body limit, causing a 413 error. v2 requests are unchanged.
  3. POST /code_suggestions/direct_access now also returns model_metadata in the v4 shape (provider, feature_setting, identifier), but only when a model is pinned through model selection. GitLab default returns no model_metadata, so a v4 direct connection lets the gateway's default_models decide the model. Self-hosted settings are excluded because their metadata carries the model API key, which must not reach the IDE. The existing model_details field is unchanged, so current v2 IDE clients see no difference.

Items 1 and 2 affect every completion that goes through Rails: all self-managed instances (direct connection is refused for online cloud licenses), Amazon Q customers, and GitLab.com fallback traffic. Item 3 prepares the language server's move of direct connections from /v2/completions to /v4/code/suggestions, where a pinned model can only be expressed as model_metadata.

Merge this MR after the AI Gateway MR gitlab-org/modelops/applied-ml/code-suggestions/ai-assist!7024 (merged), which raises the per-item code_context limit to 500 kB. Before that gateway change merges, a context item over 100 kB is rejected by the gateway with a 422 error.

References

Screenshots or screen recordings

Backend only. No UI change.

How to set up and validate locally

  1. Run a gateway at or after gitlab-org/modelops/applied-ml/code-suggestions/ai-assist!7024 (merged). Check out that branch and run gdk restart.
  2. Enable the flag: Feature.enable(:code_completion_v4_endpoint).
  3. Send POST /api/v4/code_suggestions/completions with a PAT with api scope and the header X-Gitlab-Language-Server-Version: 9.16.0, with this body: 
    {"current_file":{"file_name":"main.py","content_above_cursor":"import math\n\ndef","content_below_cursor":"\n"},"intent":"completion","stream":false,"context":[{"type":"file","name":"util.py","content":"def sub(a, b):\n    return a - b\n"}]}
    This returns 200 with a suggestion.
  4. Repeat with a 200000-character content value. This returns 200. On master the context never reaches the gateway as a component, and against a gateway without the 500 kB change the request returns 422.
  5. In the request spec, the Workhorse body for v4 now contains only prompt_components, and model_metadata when a model is pinned or self-hosted.
  6. Send POST /api/v4/code_suggestions/direct_access. With GitLab default the response has model_details and no model_metadata. Pin a model for code completions in the instance model selection settings and repeat: the response adds model_metadata with provider: gitlab, feature_setting: code_completions and the pinned identifier, while model_details is unchanged.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

RuboCop is clean on all 11 changed files. The five touched spec files (ee/spec/lib/code_suggestions/model_details/code_completion_spec.rb, ee/spec/lib/code_suggestions/prompts/code_completion/ai_gateway_code_editor_completion_spec.rb, ee/spec/lib/code_suggestions/tasks/code_completion_spec.rb, ee/spec/lib/code_suggestions/tasks/code_generation_spec.rb, ee/spec/requests/api/code_suggestions_spec.rb) pass: 337 examples, 0 failures. End-to-end tests on GDK against the gateway branch confirm: a completion with a small context item returns a real suggestion, a completion with a 207 kB context item returns a real suggestion, and a non-stream generation still works.

Edited by Mike Wronski

Merge request reports