[Feedback] Self-Hosted Duo Chat
This issue is to provide feedback on the beta of self-hosted GitLab Duo Chat. This feature enables self-managed customers to leverage Gitlab Duo features on-prem, or from within a supported private cloud environment.
Self-hosted Gitlab Duo Chat supports use of the following models on the following model platforms.
| Model Family | Model | Supported Platforms | Code completion | Code generation | GitLab Duo Chat |
|---|---|---|---|---|---|
| Mistral Codestral | Codestral 22B v0.1 | vLLM | Fully compatible | Fully compatible | Not applicable |
| Mistral | Mistral 7B-it v0.3 | vLLM | Largely compatible | Fully compatible | Not compatible |
| Mistral | Mixtral 8x7B-it v0.1 | vLLM, AWS Bedrock | Largely compatible | Fully compatible | Largely compatible |
| Mistral | Mixtral 8x22B-it v0.1 | vLLM | Fully compatible | Fully compatible | Largely compatible |
| Claude 3 | Claude 3.5 Sonnet | AWS Bedrock | Fully compatible | Fully compatible | Fully compatible |
| Claude 3 | Claude 3.7 Sonnet | AWS Bedrock | Fully compatible | Fully compatible | Fully compatible |
| GPT | GPT-4 Turbo | Azure OpenAI | Fully compatible | Fully compatible | Largely compatible |
| GPT | GPT-4o | Azure OpenAI | Fully compatible | Fully compatible | Fully compatible |
| GPT | GPT-4o-mini | Azure OpenAI | Fully compatible | Fully compatible | Largely compatible |
| Llama | Llama 3 8B | vLLM | Largely compatible | Fully compatible | Not compatible |
| Llama | Llama 3.1 8B | vLLM | Largely compatible | Fully compatible | Largely compatible |
| Llama | Llama 3 70B | vLLM | Largely compatible | Fully compatible | Not compatible |
| Llama | Llama 3.1 70B | vLLM | Largely compatible | Fully compatible | Fully compatible |
| Llama | Llama 3.3 70B | vLLM | Largely compatible | Fully compatible | Fully compatible |
Asks
Please leave feedback or questions about this feature and ping @susie.bee directly in your feedback. We're interested in learning what concerns you have or the impacts you see. The below are some suggested areas for feedback, but any feedback at all is appreciated.
- What models and model platform are you using to support self-hosted models?
- What models or model platforms would you love to see added?
- How was the process for setting up self-hosted models on your environment?
- What is working and not working for your organization?
Edited by Susie Bitters