feat: adding codestral 25.08 and fixing existing issue with model switching

Omar Qunsulrequested to merge
569738-adding-codestral-25-08 into main

What does this merge request do and why?

It's addressing this issue: gitlab-org/gitlab#569738 But there are other things that have been solved in this MR. This is a list of all of the changes:

  • Codesetral 25.08 as been added as a selectable option for Code Completion.
  • There used to be a bug, that when manually choosing Fireworks / Codestral 25.01, it was sending the request to Vertex AI instead, but also Codestral 25.01. This has been also solved in this MR.
  • When choosing Vertex Codestral 25.01, it would still send the request to Vertex. This is still working as requested.
  • Anthropic models (on Anthropic) didn't work before. But with this MR this has been fixed.

Similar Problems that are still not solved in this MR, but can be separated into another issue. But I believe it needs to be discussed

  • Anthropic Models (on VertexAI) didn't and are still not working. I tried to solve it, but it seems it needs a lot of work. So I am creating another issue to discuss this. Because maybe if they aren't working now, we can remove them instead of fixing the problem?

This indeed has already affected some customer. See here

More context about the fix

When the user changes the model from Rails settings, Rails sends the provider gitlab for any other chosen model. But the If statement that was removed in the Python code was expecting a different provider. That's why the new code checks the provider from the list of the models.yml instead of the one sent via Payload.

How to set up and validate locally

  1. Change Gitlab Rails code by adding this code to ee/app/models/ai/model_selection/instance_model_selection_feature_setting.rb. There is no config around it at the moment:
      def base_url
        return Ai::Setting.first.ai_gateway_url <---- ADD THIS LINE TEMPORARILY

        ::Gitlab::AiGateway.cloud_connector_url
      end

There is no way around it at the moment that I am aware of it. Because when a custom model is selected, it calls the Cloud Connector. But we need to test this with the local AI Gateway

  1. Choose another Fireworks Codestral model from this page. You can anything that choose anything other than the default one, and it will still work. Codestral 25.01 or the new one 25.08 should work. 25.08 deployment is set on autoscale mode, and the minimum is 0. So it might take around 2 minutes to scale up. Once this is merged, I can set the minimum to 1. Please note that currently when you choose another Fireworks/Codestral model, the result comes from Vertex instead. So that's a bug that is solved by this MR.

  2. Choose another Anthropic model. It should work now, but right now the generated code doesn't look much like Code Completion. I am planning to create another issue for this to discuss this. I don't know if this was working at some point in the past. But right now on main this is not working at all. So this MR at least makes it work to some extent.

  3. Please change back to the default model when you are done testing.

Merge request checklist

  • Tests added for new functionality. If not, please raise an issue to follow up.
  • Documentation added/updated, if needed.
  • If this change requires executor implementation: verified that issues/MRs exist for both Go executor and Node executor or confirmed that changes are backward-compatible and don't break existing executor functionality.
Edited by Omar Qunsul

Merge request reports