Central Evaluation Framework - add tests for /explain, /refactor, /tests in Chat
Problem to solve
GitLab Duo Chat currently offers distinct tools for explaining code, refactoring code, and generating tests for code. The central eval framework currently contains user written prompts (or prompts generated from user written prompts). However, the prompts behind these distinct tools are crafted by GitLab and we currently don't test how well they achieve the goal.
Proposal
Create tests for /explain
, /refactor
, /tests
. See documentation of these three features: https://docs.gitlab.com/ee/user/gitlab_duo_chat.html#explain-code-in-the-ide
Further details
Links / references
Draft: Add support for GitLab slash command (!363 - closed)
Set claude 2.1 as model for slash commands (gitlab-org/gitlab!149011 - merged)
Code Explanation / Refractoring with GitLab Data (#131 - closed)
Edited by Susie Bitters