Separate AI Gateway and Duo Workflow projects
Problem
Duo Workflow started building a new service under top-level directories such as duo_workflow
and autograph
in AI Gateway project. While these services are maintained in the same repository, they are independent at network and application level, hence this causes the following issues.
- Violation of separation of concern. e.g. chore: add Duo Workflow ai credentials (!1006 - closed) is trying to load entire AI Gateway config in Duo Workflow or vice-versa. These should be bounded to each service and the config should be isolated.
- Skewered versioning: Since AI Gateway and Duo Workflow are on the same version track, adding a change to one service effectively bumps the other service's version even though it's not changed.
- Skewered CI/CD configs: We need to separately manage the CI/CD pipelines for each service. This requires splitting the pipeline configuration and having a proper
rules:change:
for each job to trigger a pipeline for affected services only. This will be more complicated when we share the same config file.
Since Duo Workflow uses HTTP2 on gRPC instead of HTTP1.1/Websocket on FastAPI, there is no point to have these services in the same repostiory.
Proposal
- Maintain AI Gateway and Duo Workflow projects in different repositories. You can fork AI Gatway repository if you're planning to use some modules.
- Follow-up: For DRY, we create a python package for shared components, such as model instrumentor (or use git-submodules).
Old description
Options
List (not exhaustive) of options that we can consider:
- Share 3rd party AI providers bandwidth with the rest of AI features at GitLab:
- it will be required to assure that currently provisioned resources are sufficient to support Duo Workflow
- it will be required to consider unified threshold system to enforce those limits both on standard and Duo Workflow deployments of the AI Gateway
- Use dedicated bandwidth just for Duo Workflow
- We need to provision necessary resources
- We need to introduce threshold monitoring and enforcement mechanisms for Duo Wrokflow
Short term
Short term approach is that Duo Workflow for the purpose of development will receive its own dedicated bandwidth and credentials
Related
- This issue is a follow up to the @shinya.maeda question about reusing credentials, therefore bandwidth of AI Gateway with Duo Workflow
Edited by Shinya Maeda