Upgrade to Claude 3 for Code Generation
Claude 3
Anthropic just released a new version of their Claude models, Claude 3. You can read the public announcement here.
We are specifically interested in the Claude 3 Sonnet
model. However, now that Anthropic has announced multiple models, we should verify that Sonnet
is the model that best fits our needs.
We did some initial testing and found the quality was at least on par with Claude 2
and the latency was improved.
Prompt Changes
From early testing, the current prompt will mostly work. However, we need to move to the Messages API from Anthropic. Part of that involves removing the "Human" and "Assistant" parts of the prompt, which are now set with the API call.
However, we should experiment to see if we can make any prompt changes that will yield better results or faster latency.
Feature Flag
We need to put any changes behind a feature flag so we can control the rollout
Documentation
We will need to update the documentation to indicate the model we are using after we roll it out.