Mixtral 8x7B-Instruct/8x22b prompt: IDE Code Generation

Overview

Develop IDE Code Generation Prompts and supporting code for the IDE for Mixtral 8x7B-Instruct. Prompt iteration will start based on minimum prompt templates to see if scores can be improved from Validation baselines for code generation or if additional manipulation degrades performance of the models.

task	model	correctness
Code Generation	mixtral-8x-7b-instruct-01	3.63
Code Explanation	mixtral-8x-7b-instruct-01	1.795
Issue/Epic	mixtral-8x-7b-instruct-01	1.184

This assumes that all the IDEs have the same code due to a editor abstraction layer. If this is not the case, we will split issue.

Resources

• Existing prompts
• Baseline Validation Prompts
• Smart Competition: Prompt Formatting in IDE for... (gitlab-org/modelops/ai-model-validation-and-research/ai-evaluation/prompt-library#226 - closed)
• Supported editor extensions
• Prompt Engineering Guide for Mixtral 8x7B
• Code Suggestions documentation

Definition of Done

• Prompts developed for 8x7B-Instruct prompt: IDE Code Generation
• Prompts developed for 8x22b prompt: IDE Code Generation
• Prompts validated using the Evaluation Framework