Agent registry PoC
To achieve KR: https://gitlab.com/gitlab-com/gitlab-OKRs/-/work_items/5044+ We need to create a architecture design proposal.
Scope
We'll be trying to solve the following specific problems:
- Decouple prompt versioning from monolith releases
The following will be out of scope:
- Allow users to create/modify prompts
- Provide a way for the monolith to specify a prompt version when performing completion requests
In that sense, we're looking to implement more of a "prompt library" than a "prompt registry"; i.e. something closer to our CI/CD templates than to our Container Registry.
Goals
- Make the storage and fetching of prompts as transparent as possible to the monolith, such that we can iterate on or substitute the backend without needing adjustments from the clients
- Make the interface definition generic enough such that it can be expanded down the line with more capabilites while eliminating or minimizing breaking changes
In that order of ideas, we're describing the interface in terms of Agents from the get-go, to future-proof this feature and keep it aligned with our vision for AI Agents.
POC
- Add a new folder/file structure to the AIGW repository following the format
ai_gateway/agents/[usecase]/[name].yml
. Following this structure, we could have for example the following files:
ai_gateway/
agents/
chat/
base.yml
minimal.yml
root_cause_analysis/
base.yml
Each file would contain at least the following keys:
-
name
: An arbitrary string describing the agent -
prompt_templates
: An object with one or more keyed prompt templates, each implemented in jinja
An example file would look like the following:
---
name: Claude 2.1 with minimal instructions
prompt_templates:
system: |
You are a DevSecOps Assistant named 'GitLab Duo Chat' created by GitLab.
You have access only to the following tools:
<tools_list>
{%- for tool in tools %}
<tool>
<name>{{ tool.name }}</name>
<description>
{{ tool.description }}
</description>
{%- if tool.example %}
<example>
{{ tool.example }}
</example>
{%- endif %}
</tool>
{%- endfor %}
</tools_list>
assistant: |
{agent_scratchpad}
Thought:
user: |
Question: {question}
Status (as of 2024/03/13)
-
Implement local agent registry on AIGW gitlab-org/modelops/applied-ml/code-suggestions/ai-assist!773 (merged) -
Generalize AI Gateway client to any endpoint !152318 (merged) -
Move an existing feature's business logic to the AIGW using an agent from the agent registry.
Edited by Alejandro Rodríguez