Document setup for Mixtral 8x7B, Mixtral 8x22B, and GDK
Test and document a developer setup for Mixtral 8x7B and Mixtral 8x22B for use with the GDK.
Model Sources
GitLab Internal only resources
See also
- gitlab-org/modelops/ai-model-validation-and-research/ai-evaluation/prompt-library#187 (comment 1829917070)
- Include AI Gateway in GDK (gitlab-development-kit#2025 - closed)
- Self-Hosted Model Deployment Blueprint (!148599 - merged)
Defintion of Done
- Setup and test a local development environments
- Document both setups in the Developer documentation
- include a note that the Vertex Hosted Mixtral 8x7B is GitLab internal only.
Edited by Sean Carroll