<!--- Please read this! Before opening a new issue, make sure to search for keywords in the issues filtered by the "regression" or "type::bug" label: - https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=regression - https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=type::bug and verify the issue you're about to submit isn't a duplicate. ---> ### Summary When not using the default `Codestral 25.01 - Fireworks - Default` model for `Code Completions` the GitLab Duo health check fails and returns `Authentication with the AI gateway services failed: AI Gateway returned code 422: {"detail":"Validation error"}` This [same behaviour also occurs with code suggestions in the IDE](https://support.gitlab.com/hc/en-us/articles/23849833400476-GitLab-Duo-Code-Completions-fail-with-422-detail-Validation-error) when not using `Codestral 25.01 - Fireworks - Default` for Code Completions. ### Steps to reproduce 1. In a self-managed instance, navigate to `/admin/gitlab_duo/self_hosted` and change the model used for Code Completions to anything that is not `Codestral 25.01 - Fireworks - Default` 2. Navigate to `/admin/gitlab_duo` and run the health check 3. Observe 422 validation error failure 4. Switch model for Code Completion back to the default model, re-run health check and observe it succeed Note, even using `Codestral 25.01 - Fireworks` (without the `- Default`) results in the 422 validation error. ### Example Project N/A ### What is the current *bug* behavior? Health check and code completions/suggestions fail with 422 validation error when using the non-default model for code suggestions ### What is the expected *correct* behavior? 1. Health check and code suggestions should not fail for these other models or 2. The other models (like 4.5 sonnet) should not be available for Code Completions if they are not compatible ### Relevant logs and/or screenshots 25.01 Codestral Fireworks (`non-default`) {width=900 height=469} 25.01 Codestral Fireworks - Default {width=900 height=463} 4.5 Sonnet: ```ruby client = Gitlab::Llm::AiGateway::CodeSuggestionsClient.new(user) task = client.send(:task) task.body ### Output "{\"current_file\":{\"file_name\":\"test.rb\",\"content_above_cursor\":\"def hello_world\",\"content_below_cursor\":\"\"},\"model_provider\":\"gitlab\",\"model_name\":\"claude_sonnet_4_5_20250929\"}" ``` 25.01 Codestral Fireworks - Default ```ruby client = Gitlab::Llm::AiGateway::CodeSuggestionsClient.new(user) task = client.send(:task) task.body ### Output "{\"current_file\":{\"file_name\":\"test.rb\",\"content_above_cursor\":\"def hello_world\",\"content_below_cursor\":\"\"},\"prompt_version\":1,\"model_name\":\"codestral-2501\",\"model_provider\":\"fireworks_ai\"}" ``` 25.01 Codestral Fireworks (`non-default`) ```ruby client = Gitlab::Llm::AiGateway::CodeSuggestionsClient.new(user) task = client.send(:task) task.body "{\"current_file\":{\"file_name\":\"test.rb\",\"content_above_cursor\":\"def hello_world\",\"content_below_cursor\":\"\"},\"model_provider\":\"gitlab\",\"model_name\":\"codestral_2501_fireworks\"}" ``` ### Output of checks <!-- If you are reporting a bug on GitLab.com, uncomment below --> <!-- This bug happens on GitLab.com --> <!-- and uncomment below if you have /label privileges --> <!-- /label ~"reproduced on GitLab.com" --> <!-- or follow up with an issue comment of `@gitlab-bot label ~"reproduced on GitLab.com"` if you do not --> #### Results of GitLab environment info <!-- Input any relevant GitLab environment information if needed. --> <details> <summary>Expand for output related to GitLab environment info</summary> <pre> (For installations with omnibus-gitlab package run and paste the output of: `sudo gitlab-rake gitlab:env:info`) (For installations from source run and paste the output of: `sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`) </pre> </details> #### Results of GitLab application Check <!-- Input any relevant GitLab application check information if needed. --> <details> <summary>Expand for output related to the GitLab application check</summary> <pre> (For installations with omnibus-gitlab package run and paste the output of: `sudo gitlab-rake gitlab:check SANITIZE=true`) (For installations from source run and paste the output of: `sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true`) (we will only investigate if the tests are passing) </pre> </details> ### Possible fixes <!-- If you can, link to the line of code that might be responsible for the problem. --> ### Patch release information for backports If the bug fix needs to be backported in a [patch release](https://handbook.gitlab.com/handbook/engineering/releases/patch-releases) to a version under [the maintenance policy](https://docs.gitlab.com/policy/maintenance/), please follow the steps on the [patch release runbook for GitLab engineers](https://gitlab.com/gitlab-org/release/docs/-/blob/master/general/patch/engineers.md). Refer to the [internal "Release Information" dashboard](https://dashboards.gitlab.net/d/delivery-release_info/delivery3a-release-information?orgId=1) for information about the next patch release, including the targeted versions, expected release date, and current status. #### High-severity bug remediation To remediate high-severity issues requiring an [internal release](https://handbook.gitlab.com/handbook/engineering/releases/internal-releases/) for single-tenant SaaS instances, refer to the [internal release process for engineers](https://gitlab.com/gitlab-org/release/docs/-/blob/master/general/internal-releases/engineers.md?ref_type=heads). <!-- If you don't have /label privileges, follow up with an issue comment of `@gitlab-bot label ~"type::bug"` -->