Skip to content

Implement Chat prompt changes for better Claude 3 code generation result

Lesley Razzaghian requested to merge implement-suggested-duo-prompt-changes into master

What does this MR do and why?

It was noticed that Claude 3 does not perform as well as Claude 2.1 with the existing prompt for code generation.

@timzallmann experimented with the prompt and found that these changes gave better results.

This MR implements the changes he suggsted.

MR acceptance checklist

✅

Screenshots or screen recordings

Before

image

After

image

How to set up and validate locally

  1. Enable Claude 3 feature flag
Feature.enable(:ai_claude_3_sonnet)

It may be helpful to do a quick manual regression test of Duo Chat locally.

Here is a code generation example question: Write a function to find the product of first even and odd number of a given list.

The CiEditorAssistant class is also affected. Here is an example question: Please create a deployment configuration for a node.js application

Edited by Lesley Razzaghian

Merge request reports