Skip to content

Agent registry PoC

To achieve KR: https://gitlab.com/gitlab-com/gitlab-OKRs/-/work_items/5044+ We need to create a architecture design proposal.

Scope

We'll be trying to solve the following specific problems:

  • Decouple prompt versioning from monolith releases

The following will be out of scope:

  • Allow users to create/modify prompts
  • Provide a way for the monolith to specify a prompt version when performing completion requests

In that sense, we're looking to implement more of a "prompt library" than a "prompt registry"; i.e. something closer to our CI/CD templates than to our Container Registry.

Goals

  • Make the storage and fetching of prompts as transparent as possible to the monolith, such that we can iterate on or substitute the backend without needing adjustments from the clients
  • Make the interface definition generic enough such that it can be expanded down the line with more capabilites while eliminating or minimizing breaking changes

In that order of ideas, we're describing the interface in terms of Agents from the get-go, to future-proof this feature and keep it aligned with our vision for AI Agents.

POC

  1. Add a new folder/file structure to the AIGW repository following the format ai_gateway/agents/[usecase]/[name].yml. Following this structure, we could have for example the following files:
ai_gateway/
  agents/
    chat/
      base.yml
      minimal.yml
    root_cause_analysis/
      base.yml

Each file would contain at least the following keys:

  • name: An arbitrary string describing the agent
  • prompt_templates: An object with one or more keyed prompt templates, each implemented in jinja

An example file would look like the following:

---
name: Claude 2.1 with minimal instructions
prompt_templates:
  system: |
    You are a DevSecOps Assistant named 'GitLab Duo Chat' created by GitLab.

    You have access only to the following tools:

    <tools_list>
    {%- for tool in tools %}
        <tool>
            <name>{{ tool.name }}</name>
            <description>
            {{ tool.description }}
            </description>
            {%- if tool.example %}
            <example>
            {{ tool.example }}
            </example>
            {%- endif %}
        </tool>
    {%- endfor %}
    </tools_list>
  assistant: |
    {agent_scratchpad}
    Thought:
  user: |
    Question: {question}

Status (as of 2024/03/13)

Edited by Alejandro Rodríguez