AI-gateway API & architecture
AI-gateway API & architecture
This explains the AI-gateway architecture, using code-suggestions as an example.
It focuses on the design of single purpose endpoints that the AI-gateway implements to support GitLab features using AI. Similar to how Gitaly allows GitLab to access repositories.
It documents that the AI-gateway will need to support GitLab-versions 2 major versions old, and supporting multiple versions of the VSCode extension.
It also touches on how the AI gateway can be used as a proxy for AI providers for features that GitLab currrently has that call out directly to external providers.
It touches briefly on the architecture of the AI gateway with a diagram and documents the way GitLab instances can communicate with the AI gateway, including the protocol and the schema of the messages being passed around.
Not covered but mentioned in this:
- Authentication on the AI-gateway, moved to: gitlab-org/modelops/applied-ml/code-suggestions/ai-assist#177 (closed)
- Telemetry, moved to: https://gitlab.com/gitlab-org/gitlab/-/issues/415745
Rendered version: https://gitlab.com/gitlab-com/gl-infra/readiness/-/blob/bvl/ai-gateway-draft/library/ai-gateway/index.md?ref_type=heads
Part of gitlab-org/modelops/applied-ml/code-suggestions/ai-assist#161 (closed)