Skip to content

Add support for using CodeGemma instruction-tuned for code generations

Bruno Cardoso requested to merge bc/custom-models-code-gemma-it into master

What does this MR do and why?

This MR adds support for using a codegemma:7b-instruct model for code generations through the Code Suggestions API.

This is the instruction-tuned 7B version of the CodeGemma model family (ref).

Part of #461468 (closed).

Evaluation results

I've ran the code-creation eval pipeline against a local gdk instance configured to use codegemma:7b-instruct for code generations.

Mean scores Score distribution
image image
Top 3 biggest discrepancies from baseline
image

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

  1. Video on how to test it

Screen_Recording_2024-06-12_at_15.51.47

Edited by Bruno Cardoso

Merge request reports