Mixtral 8x7B-Instruct/8x22b prompt: IDE Code Generation
Overview
Develop IDE Code Generation Prompts and supporting code for the IDE for Mixtral 8x7B-Instruct. Prompt iteration will start based on minimum prompt templates to see if scores can be improved from Validation baselines for code generation or if additional manipulation degrades performance of the models.
task |
model |
correctness |
Code Generation | mixtral-8x-7b-instruct-01 | 3.63 |
Code Explanation | mixtral-8x-7b-instruct-01 | 1.795 |
Issue/Epic | mixtral-8x-7b-instruct-01 | 1.184 |
This assumes that all the IDEs have the same code due to a editor abstraction layer. If this is not the case, we will split issue.
Resources
- Existing prompts
- Baseline Validation Prompts
- Smart Competition: Prompt Formatting in IDE for... (gitlab-org/modelops/ai-model-validation-and-research/ai-evaluation/prompt-library#226)
- Supported editor extensions
- Prompt Engineering Guide for Mixtral 8x7B
- Code Suggestions documentation
Definition of Done
- Prompts developed for 8x7B-Instruct prompt: IDE Code Generation
- Prompts developed for 8x22b prompt: IDE Code Generation
- Prompts validated using the Evaluation Framework
Edited by Susie Bitters