Prompt Migration: Prepare AI Gateway to be extendable by custom models
Problem to solve
Within Prompt Migration to AI Gateway (gitlab-org&14259) we plan to start migrating prompts from GitLab Rails to AI Gateway. We need to prepare AI Gateway to be easily extendable by any model/provider and accept custom models params.
Proposal
-
Use litellm instead of the existing Anthropic client: feat(agents): use litellm in generic agents (!996 - merged) -
Extend agent types to accept model_endpoint
/model_api_key
params
Edited by Igor Drozdov