Skip to content

Refactor current Anthropic and Vertex client

Description

This issue involves modifying the Anthropic and Vertex clients to use different URLs for their requests. Instead of directly accessing the provider URLs, the clients should use URLs from a AI gateway.

Requirements

  1. Feature Flag: The changes should be implemented behind a feature flag to maintain control over the new behavior initially.

  2. URL Adjustment: The URLs used by the clients should be adjusted to align with the work being done by @shinya.maeda.

  3. Payload Structure: The payloads sent by the clients should remain unchanged. The gateway endpoints should act as transparent proxies, without modifying the payload structure.

  4. Token Adjustment: Additionally, token adjustments are required, but this work is being shared with the Cloud Connector team and will be covered in a separate issue.

Tasks

  1. Identify the current URLs used by the Anthropic and vertex clients.
  2. Implement a feature flag to control the new behavior.
  3. Update the clients to use the gateway URLs instead of the direct provider URLs.
  4. Ensure that the payloads sent by the clients remain unchanged.
  5. Coordinate with the Cloud Connector team regarding the token adjustment work.
  6. Notify the relevant stakeholders, including the Cloud Connector team, about the changes and ensure they can provide assistance if needed.
Edited by Gosia Ksionek