Skip to content

Add Codestral on Vertex support for code completion

Google has launched Codestral on Vertex as a managed service. This seems like a suitable option for code completion, so we are trying to add it as an option with a feature flag.

SDK Concerns

Codestral on Vertex is only supported by using the HTTP interface to vertex, not the vertex python SDK. Confirmed with Google that the python SDK only plans to support first party models, not other models hosted on vertex such as Anthropic/Mistral models.

The stated preference for adding new models to code completion is using litellm (which has vertex HTTP support) and only use the LangChain wrapper if the litellm model does not support the model properly.

For Anthropic on Vertex there's some kind of issue using the litellm wrapper so we're using the LangChain wrapper.

Unfortunately neither litellm nor the LangChain google client have added Codestral on Vertex support yet.

Edited by Allen Cook