Skip to content

refactor: support both Anthropic and LiteLLM model class providers

Alexander Chueshev requested to merge ac/support-anthropic-lite-llm-providers into main

What does this merge request do and why?

This MR adds support for both the ChatAnthropic and ChatLiteLLM model class providers when instantiating an agent instance.

Related discussion: #541 (closed)

How to set up and validate locally

  1. To use ChatAnthropic class, update the chat/react.yml config following:

    name: Claude 3 ReAct Chat agent
    model:
      name: claude-3-sonnet-20240229
      params:
        model_class_provider: anthropic
        temperature: 0.0
        timeout: 60
        max_tokens: 2_048
        max_retries: 1
        # Please, check `agents.config.models.ChatAnthropicParams` for the list of supported model parameters
  2. To use ChatLiteLLM class, update the chat/react.yml config following:

     model:
      name: claude-3-sonnet-20240229
      params:
        model_class_provider: lite_llm
        temperature: 0.0
        timeout: 60
        max_tokens: 2_048
        max_retries: 1
        # Please, check `agents.config.models.ChatLiteLLMParams` for the list of supported model parameters

Merge request checklist

  • Tests added for new functionality. If not, please raise an issue to follow up.
  • Documentation added/updated, if needed.

Merge request reports