feat: add v2 prompt versions to support claude 3.7 sonnet

added devopsai-powered sectiondata-science labels

Reviewer roulette

To spread load more evenly across eligible reviewers, Danger has picked a candidate for each review slot. Feel free to override these selections if you think someone else would be better-suited or use the GitLab Review Workload Dashboard to find other available reviewers.

To read more on how to use the reviewer roulette, please take a look at the Engineering workflow and code review guidelines.

Once you've decided who will review this merge request, mention them as you normally would! Danger does not automatically notify them for you.

Reviewer	Maintainer
`@mhamda` (UTC+1, 8 hours ahead of author)	`@eduardobonet` (UTC+1, 8 hours ahead of author)

If needed, you can retry the danger-review job that generated this comment.

Generated by Danger

@tlinz

Can I have some help with Duo Chat to ensure the following are the correct model configurations for the tool Duo Chat tool?

Proper labels assigned to this merge request. Please ignore me.

@nateweinshenker - please see the following guidance and update this merge request.

	1 Error
	Please add typebug typefeature, or typemaintenance label to this merge request.

added 1 commit

d548a8d7 - chore: add unit primitive

Compare with previous version

changed the description

added 1 commit

72c48526 - fix: unit primitive change

Compare with previous version

added typemaintenance label

resolved all threads

requested review from @jessieay

@nateweinshenker can you fill out "## How to set up and validate locally" ?

@jessieay

Update the issue description.

@jessieay Could you please review this merge request?

This change adds a new model configuration file. The follow-up change will set default Duo Chat definitions behind a 2.0.0 model version. We're implementing this approach for a gradual rollout of the new prompt definitions.

Here's the associated issue tracking this change.

index 3cfa1423..a97fc20b 100644
--- a/ai_gateway/chat/container.py
+++ b/ai_gateway/chat/container.py
@@ -5,6 +5,7 @@ from dependency_injector import containers, providers
 from ai_gateway.chat.agents import ReActAgent, TypeAgentEvent
 from ai_gateway.chat.executor import GLAgentRemoteExecutor, TypeAgentFactory
 from ai_gateway.chat.toolset import DuoChatToolsRegistry
+from ai_gateway.feature_flags import FeatureFlag, is_feature_enabled
 
 if TYPE_CHECKING:
     from ai_gateway.prompts import BasePromptRegistry
@@ -18,6 +19,9 @@ def _react_agent_factory(
     prompt_registry: "BasePromptRegistry",
 ) -> TypeAgentFactory[TypeAgentEvent]:
     def _fn(**kwargs) -> ReActAgent:
+        if is_feature_enabled(FeatureFlag.ENABLE_ANTHROPIC_CLAUDE_3_7):
+            return prompt_registry.get("chat/react", "^2.0.0", **kwargs)
+
         return prompt_registry.get("chat/react", "^1.0.0", **kwargs)

The diff will have usfeature flag check for enabling Claude 3.7, which will use the new 2.0.0 version of the chat/react prompt when enabled.

changed the description

added 1 commit

3efb43b3 - chore: fix doc search prompt

Compare with previous version

changed this line in version 5 of the diff

Yes, I'm for having a distinct name.

Question: why does this have a different UP from the 1.0 version?

@jessie thanks for the catch!

changed this line in version 5 of the diff

Question: should this include max_tokens and other params like 1.0?

@jessieay We don't need to include the max_token because the prompt file overrides the configuration file, given that it's specified last in the dict merge, and the last keys take precedence. E.g.:

>>> {**{'a': 1, 'b': 2}, **{'b': 3, 'c': 4}}
{'a': 1, 'b': 3, 'c': 4}

Hi @nateweinshenker - I need to take my dog to the vet right now but left some initial feedback here. Can you explain why these prompt templates would look different from the 1.0.0 versions?

@jessieay Part of this was due to Dogfooding Duo Workflow to help speed up making these prompt versions. I didn't do enough combing through the prompt differences before placing it in an MR. Luckily, they are caught by our unit tests https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/jobs/9232041649.