### Release notes Semantic Code Search self-hosted embedding configuration now supports customers whose environment routes AI traffic through a corporate LiteLLM proxy that fronts a cloud embedding provider (Vertex, Bedrock, Azure OpenAI) under that provider's native LiteLLM prefix, without requiring proxy-side aliasing or exposing GitLab admins to opaque errors. ### Problem to solve The self-hosted embedding model flow (introduced in 19.1, still beta) works cleanly when the customer connects the AI Gateway directly to a supported embedding provider (Azure OpenAI, Fireworks, Bedrock with credentials mounted, and so on). It breaks in a common enterprise pattern: an intermediate corporate LiteLLM proxy that fronts a cloud embedding provider under that provider's native prefix, for example a proxy that registers `text-embedding-005` under `litellm_provider: vertex_ai-embedding-models` and does not also expose an OpenAI-compatible embedding route. In that setup, none of the documented Model identifier prefixes work: - `text-embedding-005` (bare) resolves to `custom_llm_provider=custom_openai` (the AI Gateway's documented fallback). LiteLLM rejects with `Unmapped LLM provider for this endpoint. You passed model=text-embedding-005, custom_llm_provider=custom_openai`. `custom_openai` is a valid provider in LiteLLM's chat/completion routing table but not in the embeddings routing table. - `custom_openai/text-embedding-005` produces the same rejection. - `vertex_ai/text-embedding-005` triggers LiteLLM's Vertex client, which authenticates directly with Google Cloud via Application Default Credentials. This bypasses the customer's proxy entirely (defeating the reason for routing through it) and fails when the AI Gateway container has no GCP credentials mounted. - `openai/text-embedding-005` is correct in principle, but only works if the proxy already exposes the model under a documented OpenAI-compatible `/v1/embeddings` route with that model name. Proxies that only have the native provider mapping return 404. The failure surfaces in the Admin UI as `AI Gateway returned code 500: Unknown error` during the "Test connection" step. Admins have no way to see the underlying LiteLLM rejection without SSHing to the AI Gateway container and reading logs. Once the actual error is visible, the resolution requires proxy-side changes the GitLab admin does not always control. The immediate workaround is to have the proxy team add a second `model_list` entry that aliases the same backend under an OpenAI-shaped name. That works but requires shared-proxy modifications that many enterprise platform teams gate through separate change control. ### Proposal Three changes, in priority order: 1. **Documentation clarity.** Update the [Add a self-hosted model prefix table](https://docs.gitlab.com/administration/gitlab_duo_self_hosted/configure_duo_features/#add-a-self-hosted-model) to make explicit which prefixes work for the EMBEDDING model family. The current table implies uniformity across model families, but LiteLLM's embedding provider set is a strict subset of its completion set (see [LiteLLM embedding docs](https://docs.litellm.ai/docs/embedding/supported_embedding)). Call out the `custom_openai` gap and the Vertex/Bedrock direct-auth requirement. 2. **Surface the real error at the UI layer.** When "Test connection" on the Add self-hosted model form fails because LiteLLM rejects the provider mapping, pass the underlying LiteLLM error message back through to the UI instead of collapsing it to `500 Unknown error`. Admins currently need container-level access to diagnose a configuration problem. 3. **Raw-passthrough embedding mode (optional but strategic).** Add a checkbox on the Add self-hosted model form (or a distinct sub-family) that tells the AI Gateway to skip LiteLLM's provider mapping and POST directly to `<api_base>/embeddings` with `{"model": "<Model identifier>", "input": [...]}`. This would let any OpenAI-compatible embedding endpoint (including corporate LiteLLM proxies fronting Vertex/Bedrock/Azure under native prefixes) work without requiring the proxy-side alias. Conceptually analogous to how `custom_openai` unblocks arbitrary OpenAI-compatible chat endpoints today. ### Intended users - Enterprise administrators configuring GitLab Duo Self-Hosted with a corporate AI gateway that gates all model traffic for regulatory, compliance, or centralised-billing reasons. - Common examples: LiteLLM proxies fronting Vertex, Bedrock, or Azure OpenAI where the proxy uses each provider's native LiteLLM prefix rather than exposing OpenAI-shaped routes. - These customers explicitly do not want to mount cloud provider credentials on the AI Gateway container, since doing so defeats the purpose of routing through the proxy. ### Feature Usage Metrics - Number of self-hosted embedding model configurations created with the EMBEDDING family (already trackable via `Ai::SelfHostedModel` records). - If the raw-passthrough mode ships, a distinct flag on those records to measure adoption of that path versus the LiteLLM-routed path. - Test-connection failure rate on the Add self-hosted model form, broken down by the underlying LiteLLM error class, would help quantify the "opaque error" pain point independently of this proposal. ### Does this feature require an audit event? No. This proposal changes how an existing admin-configured model is routed to a backend provider. The existing admin audit events for creating or updating a self-hosted model already cover the configuration change; the routing behaviour itself is not a separately auditable action. ### References - Semantic code search admin docs: https://docs.gitlab.com/administration/semantic_code_search/#self-hosted-models - Add a self-hosted model prefix table: https://docs.gitlab.com/administration/gitlab_duo_self_hosted/configure_duo_features/#add-a-self-hosted-model - Embedding LiteLLM adapter in AI Gateway: https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/blob/main/ai_gateway/models/v2/embedding_litellm.py - LiteLLM embedding provider list: https://docs.litellm.ai/docs/embedding/supported_embedding - LiteLLM proxy config docs: https://docs.litellm.ai/docs/proxy/configs - Feature MR introducing self-hosted EMBEDDING family: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/236450 - Feature MR for feature-settings sync with reference proxy config: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/240779