Configuration option to select model
Add instance-level Configuration option to select either the default LLM (Vertex / Anthropic) by Duo feature
- the ability to configure offered (GA) Duo features to the offered OS models
- for each feature, a drop down menu of the available models
- user is prompted to input necessary fields (authentification, endpoint, versioning, resource allocation, etc)
- presentation of clear documentation on each model, use case, and how each model fits to a use case based on validation
- upon selection of a model for a feature, validated prompts will be auto configured specific to that model and feature.
- Propose top-level group for this feature
See also
See also Configuration in Self-Hosted Model Deployment Blueprint (!148599 - merged) and https://gitlab.com/gitlab-org/gitlab/-/blob/a1879fad78ab5ea76f61d172184274a52283f8c1/doc/architecture/blueprints/custom_models/index.md
graph LR
a1 --> c1
a2 --> b1
b1 --> c1
b3 --> b1
b4 --> b1
c1 --> c2
c2 --> c3
c3 --> d1
d1 --> d2
subgraph "User"
a1[IDE Request]
a2[Web / CLI Request]
end
subgraph "Self-Managed GitLab"
b1[GitLab Duo Feature] <--> b2[Model & Feature-specific<br/>Prompt Retrieval]
b3[GitLab Duo Feature<br/>Configuration]
b4[LLM Serving Config]
end
subgraph "Self-Hosted AI Gateway"
c1[Inbound API interface]
c2[Model routing]
c3[Model API interface]
end
subgraph "Self-Hosted LLM"
d1[LoadBalancer]
d2[GPU-based backend]
end
Dropdown
There will be a dropdown for each Duo Feature. The following options will be available:
- Default Model (Vertex or Anthropic)
- Mistral 7B
- Mixtral 8x7B
- Feature Inactive
Definition of Done
-
Backend storage for choices -
REST API for AI Gateway to access the settings -
Frontend -
Add to menu
Edited by Susie Bitters