Skip to content

refactor: generalize Agent instrumentalization

Alejandro Rodríguez requested to merge refactor-agent-instrumentator into main

What does this merge request do and why?

refactor: generalize Agent instrumentalization

In !1067 (merged) we added instrumentalization for Agents using litellm, and left other providers pending. In #553 we discussed how to have a single implementation for all providers, which we realize here.

How to set up and validate locally

  • Start (or restart) AI Gateway
  • Get inference metrics (should be empty): curl localhost:8082/metrics | grep model
  • Make an endpoint call that uses an Agent, e.g.:
curl -X 'POST' \
   'http://localhost:5052/v2/chat/agent' \
   -H 'accept: application/json' \
   -H 'Content-Type: application/json' \
   -d '{"prompt": "What is your name?",
        "options": {
            "agent_scratchpad": {
                "agent_type": "react",
                "steps": []
            },
            "chat_history": []
        }}'
  • Get inference metrics again (should reflect the previous inference call): curl localhost:8082/metrics | grep model

Merge request checklist

  • Tests added for new functionality. If not, please raise an issue to follow up.
  • Documentation added/updated, if needed.

Merge request reports