Send review context to the GitLabDuo mention classifier

What does this MR do and why?

When Duo Code Review requests changes and the author replies in the review thread with "@GitLabDuo … please approve", Duo answers that it can't approve because it has read-only access. Duo is the one that blocked the merge request, so the author has to bypass the change request or re-add Duo as a reviewer.

The cause is that the mention intent classifier only gets the comment text. "Please approve" with no context does not look like a review request, so the mention goes to the read-only mention assistant.

Behind the duo_mention_intent_context flag, the classifier also gets:

  • The earlier messages in the thread (the last 5, each cut to 500 characters).
  • If a Duo review started the thread.
  • The Duo reviewer state on the merge request.

This data comes from a few small queries and no extra LLM call.

The AI Gateway MR (gitlab-org/modelops/applied-ml/code-suggestions/ai-assist!7756 (merged)) adds that prompt version. That MR must be deployed before the flag is enabled. If the AI Gateway does not have version 2.0.0, the classifier request fails and the mention falls back to legacy Duo Chat.

References

Screenshots or screen recordings

I took these on GDK. Before: duo_mention_intent_context off, and the AI Gateway on main. After: the flag on, and the AI Gateway on gitlab-org/modelops/applied-ml/code-suggestions/ai-assist!7756 (merged).

1. The author asks for approval in the thread of a Duo review

This is the case from #635111. Duo requested changes because the code used eval. The author fixed it and replied in the review summary thread: @GitLabDuo the config is now loaded with YAML.safe_load. Please approve.

Click to expand
Before After
Duo answers in chat that it can't approve. Its change request stays, so the merge request is still blocked. The reply starts a new Duo review. The review finds no new issues, and Duo approves the merge request.
s1_before s1_after_reply

s1_after_approved

2. The author asks Duo to approve a merge request that Duo did not review

The author adds @GitLabDuo approve this to a merge request where Duo is not a reviewer. This is not a review request, so it still goes to the chat flow, and no review starts.

Click to expand
Before After
Duo says it can't approve and tells the author to approve the merge request themselves. Duo says it can't approve, and tells the author how to start a new Duo review, which can approve the merge request.
s2_before s2_after

How to set up and validate locally

Set up GDK with this branch in gitlab, and with the AI Gateway MR branch in gitlab-ai-gateway.

  1. Make Rails call your local AI Gateway for prompt requests. In the GDK root env.runit, add export DEVELOPMENT_AI_GATEWAY_URL=http://gdk.test:5052. Then restart the services:

    gdk restart gitlab-ai-gateway duo-workflow-service rails-background-jobs
  2. In a Rails console, turn on Duo review decisions and this flag for a project (for example gitlab-duo/test). That project must have Duo Code Review (Agent Platform) turned on.

    project = Project.find_by_full_path('gitlab-duo/test')
    Feature.enable(:duo_code_review_decisions, project)
    Feature.enable(:duo_mention_intent_context, project)
  3. Create a merge request with a clear problem, for example a method that calls eval(File.read(path)). Add @GitLabDuo as a reviewer. Wait for the review: Duo posts findings and a summary, and the reviewer state is Requested changes.

  4. Fix the problem with a new commit. Reply in the thread of the Duo review summary: @GitLabDuo this is fixed now. Please approve.

    • Expected: a new Duo review starts ("requested review from @GitLabDuo"). It finds no new issues, and Duo approves the merge request.
    • In log/llm.log, the request to /v2/prompts/classify_code_review_mention_intent has "prompt_version":"^2.0.0", and the inputs include thread, thread_started_by_duo_review: true and duo_review_state: "requested_changes".
  5. In a Duo finding thread, ask a question: @GitLabDuo why is YAML.safe_load safer than eval?

    • Expected: Duo answers in the thread. No new review starts.
  6. On a different merge request where Duo is not a reviewer, add the comment @GitLabDuo approve this.

    • Expected: no review starts and Duo does not approve. Duo replies that it can't approve, and tells you how to request a review.
  7. Turn off the flag (Feature.disable(:duo_mention_intent_context, project)). Then reply @GitLabDuo please approve in a Duo review thread.

    • Expected: the old behavior. The classifier request has "prompt_version":"^1.0.0" and only message, and Duo answers in the thread.
  8. Remove the line you added to env.runit, and restart rails-background-jobs.

I did these steps on GDK. Approval works when the author fixes or answers every open finding. If one finding is still open, the new review sets the state to Reviewed (this removes the change request), but does not approve. That is the decision of the review, not a routing problem.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Wanderson Policarpo

Merge request reports

Loading
Loading