Skip to content

Conform to Anthropic prompt format conditions

Alejandro Rodríguez requested to merge anthropic-prompt-formatting into master

What does this MR do and why?

Conform to Anthropic prompt format conditions

We've been seeing errors with message prompt must start with "\n\nHuman:" turn after an optional system prompt because our conversation truncation logic would sometimes leave an Assistant: message as the beginning of the prompt. To avoid this, we only append to the prompt conversation blocks that start with a Human: message. We take the chance to follow Anthropic's suggestion of ensuring there are two line breaks before every Human: message.

Ref: https://docs.anthropic.com/claude/reference/prompt-validation

Part of #441101 (closed)

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

It's cumbersome to force this behavior naturally, because you have to account for the rest of the prompt length, and the length of the assistant responses, which is of course variable.

Merge request reports