Refactor current Anthropic and Vertex client
Description
This issue involves modifying the Anthropic
and Vertex
clients to use different URLs for their requests. Instead of directly accessing the provider URLs, the clients should use URLs from a AI gateway.
Requirements
-
Feature Flag: The changes should be implemented behind a feature flag to maintain control over the new behavior initially.
-
URL Adjustment: The URLs used by the clients should be adjusted to align with the work being done by @shinya.maeda.
-
Payload Structure: The payloads sent by the clients should remain unchanged. The gateway endpoints should act as transparent proxies, without modifying the payload structure.
-
Token Adjustment: Additionally, token adjustments are required, but this work is being shared with the Cloud Connector team and will be covered in a separate issue.
Tasks
- Identify the current URLs used by the
Anthropic
andvertex
clients. - Implement a feature flag to control the new behavior.
- Update the clients to use the gateway URLs instead of the direct provider URLs.
- Ensure that the payloads sent by the clients remain unchanged.
- Coordinate with the Cloud Connector team regarding the token adjustment work.
- Notify the relevant stakeholders, including the Cloud Connector team, about the changes and ensure they can provide assistance if needed.