Mixtral 8x7B prompt: Code Generation
For our MVP, and any other features moving forward based on Mistral, we will need to customize prompts to the models.
Existing prompts can be found here https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/lib/gitlab/llm/chain/tools?ref_type=heads
Baselined Information
Mixtral 8x7B show promise for use in code generation, with 3.63 (out of 4) score for code generation and beating out Gemini 1.0 and Claude 2.1.
Code Generation Independent LLM Judge results
Code Generation Collective LLM Judge results
Definition of Done
- Prompts developed for Mixtral 7x8B
- Code Generation
- Prompts validated using the Evaluation Framework
Edited by Susie Bitters