Agents prompts for multiple/different models

Overview

Currently, we load a single YAML file per feature and that file defines which model is used and template is used. For example:

• Chat agent gets the file from registry
• A model is chosen based on the config. Currenty, Anthropic only, potentially any model when feat(agents): use litellm in generic agents (!996 - merged) is merged

react.yml has hash format:

---
name: Claude 3 ReAct Chat agent
provider: anthropic
model: claude-3-sonnet-20240229
prompt_template:
  system: |
    System prompt content
  user: |
    User prompt content
  assistant: |
    Assistant prompt content
stop:
  - "Observation:"

Problem to solve

The current format of the YAML file implies a single model per feature. However, if we want to experiment with multiple models or support custom models, we need to handle multiple potential models per feature.

Proposal

1. Store multiple agent definitions per file

-
  name: Claude 3 ReAct Chat agent
  provider: anthropic
  model: claude-3-sonnet-20240229
  prompt_template:
    system: |
      System prompt content
    user: |
      User prompt content
    assistant: |
      Assistant prompt content
  stop:
    - "Observation:"
-
  name: Mistral ReAct Chat agent
  provider: mistral
  model: mistral
  prompt_template:
    system: |
      System prompt content
    user: |
      User prompt content
    assistant: |
      Assistant prompt content
  stop:
    - "Observation:"

When we start to support an extra model, its definition/prompt is added to the file. The default definition is chosen by convention (like the first value) or an extra param (like primary key).

2. Different file for every feature/model pair

Model definitions are stored in different files and the name of the model is reflected in the path: chat/react/claude-3-sonnet-20240229.yml. If the file is not found, chat/react/base.yml is used.