refactor: generalize Agent instrumentalization
What does this merge request do and why?
refactor: generalize Agent instrumentalization
In !1067 (merged) we added instrumentalization for Agents using litellm, and left other providers pending. In #553 we discussed how to have a single implementation for all providers, which we realize here.
How to set up and validate locally
- Start (or restart) AI Gateway
- Get inference metrics (should be empty):
curl localhost:8082/metrics | grep model
- Make an endpoint call that uses an Agent, e.g.:
curl -X 'POST' \
'http://localhost:5052/v2/chat/agent' \
-H 'accept: application/json' \
-H 'Content-Type: application/json' \
-d '{"prompt": "What is your name?",
"options": {
"agent_scratchpad": {
"agent_type": "react",
"steps": []
},
"chat_history": []
}}'
- Get inference metrics again (should reflect the previous inference call):
curl localhost:8082/metrics | grep model
Merge request checklist
-
Tests added for new functionality. If not, please raise an issue to follow up. -
Documentation added/updated, if needed.