Implement AI Gateway integration for Summarize Proposed Changes in a MR
Overview
We originally built this AI feature independently of the AI Gateway (AI GW). In order to mature the feature to GA, we need it to support self-managed instances.
More context: AI Gateway as the Sole Access Point for Monolit... (&13024)
Proposal
- Collaborate directly with groupai framework on this work, see @oregand's message https://gitlab.com/gitlab-org/gitlab/-/issues/444274#note_1888502462:
- "We'll begin with migrating a pilot feature(we expected this to be the vun feature/s) to this new setup and then guide each team through the process of creating and implementing their respective endpoints."
- Move Summarize Proposed Changes in a MR to use the AI GW instead of a direct call to the 3rd party AI provider
- Use proxy endpoints as described in ADR 002: Exposing proxy endpoints to AI providers (!150622 - merged)
- This will be done by groupai framework by refactoring the Vertex client: Refactor current Anthropic and Vertex client (#458207)
Notes
Model used: Vertex, via Gitlab::Llm::Vertex::Client
Contacts
- Engineer DRI:
@iamphill
- PM (for groupcode review):
@phikai
- EM (for groupcode review):
@francoisrose
(backend)@andr3
(frontend)
Edited by François Rosé