Skip to content

Implement Duo Code Review evaluation endpoint

What does this MR do and why?

New endpoint is POST /api/v4/duo_code_review/evaluations.

This endpoint allows LLM evaluation tools to get the model's response using the prompt and model we use for Duo Code Review.

We need this because we don't really need to evaluate the entire feature including the review publishing and other stuff. The LLM evaluation tools will also be sending the diff, new_path and hunk to be reviewed as inputs.

Initial datasets can be found in https://gitlab.com/gitlab-org/modelops/ai-model-validation-and-research/ai-evaluation/prompt-library/-/ml/models/1000644/versions/1000857. Each row contains diff, new_path, hunk and other metadata. We are only using diff, new_path, and hunk for now.

This changes the prompt class as well to accept diff, new_path and hunk. The prompt needs to change since we won't be generating the new and old hunk lines but will be reviewing the entire raw hunk instead.

This is still behind the ai_review_merge_request feature flag.

MR that adds duo-code-review evaluate command to ELI5 tool: gitlab-org/ai-powered/eli5!74 (merged).

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

How to set up and validate locally

  1. Setup local environment for AI features (doc).
  2. Enable ai_review_merge_request feature flag.
  3. Make a POST request to api/v4/duo_code_review/evaluations along with the following params: new_path, diff, hunk.
  4. You receive a JSON response something like: {"review":"Model response"}.

Related to #469095 (closed)

Edited by Patrick Bajao

Merge request reports

Loading