Serve Vertex Codestral as the code completion default behind an ops flag

Mike Wronskirequested to merge
614058-vertex-codestral-code-completion-default into master

What does this MR do and why?

This MR adds an ops feature flag, use_vertex_codestral_code_completion. The flag defaults to off. With the flag off, nothing changes.

When the flag is on, the SaaS code completion default model switches from Fireworks-hosted Codestral 25.08 (fireworks_ai / codestral-2508) to Vertex-hosted Codestral 25.08 (gitlab / codestral_2508_vertex).

The change covers the two paths the monolith controls:

  • The /code_suggestions/direct_access payload (model_details).
  • The v2 completions proxy body.

The v4 path resolves the model in the AI gateway. This MR does not change the v4 path. That change is tracked in https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/work_items/2714.

Both decision points live in ee/lib/code_suggestions/model_details/code_completion.rb, in current_model and saas_primary_model_class. One shared predicate checks the flag with the current_user actor.

Pinned models (model selection), self-hosted models, and Amazon Q are not affected. The incident failover flag (incident_fail_over_completion_provider) is not affected, and failover still takes precedence.

The flag is added to EXCLUDED_FEATURE_FLAGS in the GDK Duo setup task. Local AI gateways have no Vertex credentials.

The linked issue task list mentions factory updates. No factory changes were needed, because no factory references the default model.

Resolves https://gitlab.com/gitlab-org/gitlab/-/issues/614058. Part of epic https://gitlab.com/groups/gitlab-org/modelops/ai-model-validation-and-research/ai-evaluation/-/work_items/60.

Rollout

Rollout issue: #619299

Do not enable this flag before both of these are done:

  1. The AI gateway fix for the v2 asia-region guard _allow_vertex_codestral. This guard currently returns HTTP 500 for gitlab / codestral_2508_vertex requests on asia pods.
  2. The Vertex quota raise, tracked in https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/work_items/2736.

To revert, run /chatops run feature set use_vertex_codestral_code_completion false. The v2 proxy traffic reverts immediately. Direct-access IDE clients converge within the direct-access token lifetime, about one hour.

Docs are updated in a separate MR when the flag is enabled.

How to set up and validate locally

  1. In the rails console, run Feature.enable(:use_vertex_codestral_code_completion).
  2. Call POST /api/v4/code_suggestions/direct_access. The response model_details shows model_provider: gitlab and model_name: codestral_2508_vertex.
  3. Request a code completion from a connected IDE, with code_completion_v4_endpoint disabled. The v2 proxy body shows model_provider: gitlab and model_name: codestral_2508_vertex.
  4. Disable the flag. Both payloads return fireworks_ai / codestral-2508 again.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Merge request reports