Update AI Core Infra with functional teams

|------|-----------------|

| [Agent Foundations](/handbook/engineering/ai/agent-foundations/) | Agentic observability / Reusable Agentic components / Duo workflow service |

| [AI Coding](/handbook/engineering/ai/ai-coding/) | Code Suggestions, Duo Code Review, code-related slash commands (/explain, /refactor, /tests, /fix), Semantic Indexing, Duo Context Exclusion, Repository X-Ray  |

| [AI Framework](/handbook/engineering/ai/ai-framework/) | Abstraction Layer / AI Gateway for LLM integration to the application (GitLab Chat, Code Suggestions and other AI capabilities) |

| [AI Framework](/handbook/engineering/ai/ai-framework/) (formerly Model Validation) | Custom feature evaluators, evaluation support, automated evaluation tooling |

| [AI Core Infra](/handbook/engineering/ai/ai-core-infra/) | Abstraction Layer / AI Gateway for LLM integration to the application (GitLab Chat, Code Suggestions and other AI capabilities) |

| [AI Core Infra](/handbook/engineering/ai/ai-core-infra/) (formerly Model Validation) | Custom feature evaluators, evaluation support, automated evaluation tooling |

| [Duo Chat](/handbook/engineering/ai/duo-chat/)  | GitLab Chat for VS Code and WebIDE  |

| [Custom Models](/handbook/engineering/ai/custom-models/) | Model Selection (model lifecycle, selection engine & UI) / Health & Connectivity (Duo Health Check, setup, connectivity) / Gateway Services (Prompt Registry, Events tracking, AIGW billing) |

| [Editor Extensions: VS Code](/handbook/engineering/ai/editor-extensions-vscode/) | Maintains the GitLab Workflow VS Code Extension ([maintainers](https://gitlab-org.gitlab.io/gitlab-roulette/?currentProject=gitlab-vscode-extension&mode=show&hidden=reviewer)) & [Web IDE](https://gitlab.com/gitlab-org/gitlab-web-ide) extensions and the [language server](https://gitlab.com/groups/gitlab-org/-/epics/2431). Also contributes with UX improvements for Code Suggestions within GitLab Workflow. |

---

title: AI Core Infra Group

description: "The AI Core Infra group is focused on how to support other product groups at GitLab with the AI Abstraction Layer, and GitLab AI feature development."

---

## Overview

The AI Core Infra group owns infrastructure, reliability improvements in Duo Workflow Service and AI gateway, model evaluation tooling that every AI feature depends on as well as semantic code search.

## Team Members

The following team members are permanent members of the AI Core Infra Group:

{{< team-by-manager-slug manager="martin-w" role="AI Framework" >}}

## Stable Counterparts

The following members of other functional teams are our stable counterparts:

| **Name**                                                 | **Role** |

| ---------------------------------------------------------| ------------------------------------------------------------------------------- |

| [Ashraf Khamis](/handbook/company/team#ashrafkhamis)     | [Senior Technical Writer](/job-description-library/product/technical-writer/) |

## Functional Teams

1. [Duo Service Infra](duo-service-infra/) - Self-hosted AIGW as well as observability and reliability improvements for AI Gateway and Duo Workflow Service

1. [Model Evaluation Infra](model-evaluation-infra/) - CEF framework designed to provide a centralized solution for evaluating AI features at GitLab

1. [Semantic Code Search](semantic-code-search/) - GitLab's RAG implementation for providing semantic understanding of code repositories

## How we work

Please visit the [How We Work](how-we-work/) page for the details on working processes, meetings, and the rest.

---

title: AI Core Infra:Duo Service Infra

description: "The Duo Service Infra functional team within AI Core Infra, focused on reliability improvements and observability for AI Gateway and Duo Workflow Service."

---

## Overview

The Duo Service Infra team is part of the AI Core Infra organization, with a focus on reliability and observability for AI Gateway and Duo Workflow Service.

This page is a work in progress.

## Key Information

| | |

|---|---|

| **Slack Channel** | `#f_duo-service-infra` |

| **Stage Label** | `devops::ai platform` |

| **Group Label** | `group::ai core infra` |

| **Category Labels** | `category:duo service infra` |

## Team Meetings

1. **Duo Service Infra Weekly Sync**

   * **When:** Every Tuesday, 3PM UTC

   * **What:** A weekly sync covering status updates, ad-hoc work, and progress on long-term initiatives.

## Dashboards

1. [DWS Log-based dashboard](https://dashboards.gitlab.net/goto/efqoxvuthbwu8a?orgId=1)

1. [AI Gateway SLIs](https://dashboards.gitlab.net/d/ai-gateway-main/ai-gateway-overview?orgId=1)

1. [Duo Workflow Service SLIs](https://dashboards.gitlab.net/d/duo-workflow-svc-main/duo-workflow-svc3a-overview?orgId=1)

1. [LLM Sidekiq completions](https://dashboards.gitlab.net/d/sidekiq-main/sidekiq-overview?orgId=1)

1. [Sentry via CompletionWorker](https://new-sentry.gitlab.net/organizations/gitlab/issues/?query=is%3Aunresolved++CompletionWorker&referrer=issue-list&statsPeriod=14d)

1. [Sentry via Feature Category](https://new-sentry.gitlab.net/organizations/gitlab/issues/?query=is%3Aunresolved+feature_category%3Aai_abstraction_layer&referrer=issue-list&statsPeriod=24h)

1. [Chat REST API Error Ratio](https://log.gprd.gitlab.net/app/r/s/lDEwi)

1. [ITPM per model](https://dashboards.gitlab.net/goto/-O0w_rsHg?orgId=1)

1. [Requests per provider](https://dashboards.gitlab.net/goto/Ta-BL_-NR?orgId=1)

1. [Error budgets](https://dashboards.gitlab.net/d/product-ai-powered_error_budget/product-error-budgets-ai-powered?orgId=1)

---

title: "How We Work"

description: "How we work, communicate, and run things day-to-day in AI Core Infra"

---

## Backlog Refinement

Every week the engineering team completes a backlog refinement process

to review upcoming issues. The goal of this effort is for all issues to have a

weight so we can more accurately plan each milestone using the estimated

capacity for the team and the estimated issue weights.

In addition to this backlog refinement process, engineers on the team can add

weights to any issues that are straight-forward and do not need backlog

refinement.

This process happens in three steps.

### Step 1: Identifying Issues for Refinement

The engineering manager picks issues to refine each week. We aim for 5 issues in total.

If you think an issue is a good candidate, just mention it in the issue itself

We try to keep refinements themed to avoid too much context switching. Good

places to look for candidates:

* [Infradev Issues](https://gitlab.com/groups/gitlab-org/-/work_items?sort=created_date&state=opened&label_name%5B%5D=group%3A%3Aai%20core%20infra&label_name%5B%5D=infradev&first_page_size=20)

* [Issues scheduled for the next milestone without weight](https://gitlab.com/groups/gitlab-org/-/work_items?sort=created_date&state=opened&label_name%5B%5D=group%3A%3Aai%20core%20infra&weight=None&milestone_title=Upcoming&first_page_size=100)

* [Security Issues](https://gitlab.com/groups/gitlab-org/-/issues?scope=all&state=opened&label_name[]=group%3A%3Aai%20core%20infra&label_name[]=security&weight=None)

* [Missed-SLO](https://gitlab.com/groups/gitlab-org/-/work_items?sort=created_date&state=opened&label_name%5B%5D=group%3A%3Aai%20core%20infra&label_name%5B%5D=SLO%3A%3AMissed&first_page_size=100)

* [Approaching-SLO](https://gitlab.com/groups/gitlab-org/-/work_items?sort=created_date&state=opened&label_name%5B%5D=group%3A%3Aai%20core%20infra&label_name%5B%5D=SLO%3A%3ANear%20Miss&first_page_size=100)

* Bugs

  * [Rails](https://gitlab.com/groups/gitlab-org/-/work_items?sort=created_date&state=opened&label_name%5B%5D=group%3A%3Aai%20core%20infra&label_name%5B%5D=type%3A%3Abug&weight=None&first_page_size=20)

  * [AIGW/DWS](https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/work_items?sort=created_date&state=opened&label_name%5B%5D=group%3A%3Aai%20core%20infra&label_name%5B%5D=type%3A%3Abug&weight=None&first_page_size=20)

  * [CEF](https://gitlab.com/groups/gitlab-org/modelops/ai-model-validation-and-research/ai-evaluation/-/work_items?sort=created_date&state=opened&label_name%5B%5D=group%3A%3Aai%20core%20infra&label_name%5B%5D=type%3A%3Abug&weight=None&first_page_size=20)

* Issues without weight

  * [Rails](https://gitlab.com/groups/gitlab-org/-/work_items?sort=created_date&state=opened&label_name%5B%5D=group%3A%3Aai%20core%20infra&weight=None&first_page_size=100)

  * [AIGW/DWS](https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/work_items?sort=created_date&state=opened&label_name%5B%5D=group%3A%3Aai%20core%20infra&weight=None&first_page_size=100)

  * [CEF](https://gitlab.com/groups/gitlab-org/modelops/ai-model-validation-and-research/ai-evaluation/-/work_items?sort=created_date&state=opened&label_name%5B%5D=group%3A%3Aai%20core%20infra&weight=None&first_page_size=100)

Once selected, the engineering manager applies the `ai-framework::ready for next refinement` label and uses

the [Refinement Bot](https://gitlab.com/gitlab-org/ai-powered/ai-framework/refinement-bot)

to generate a refinement issue with all the candidates bundled together.

### Step 2: Refining Issues

Over the week, each engineer on the team will look at the list of issues

selected for backlog refinement. [Current backlog refinement issues](https://gitlab.com/groups/gitlab-org/-/work_items?sort=created_date&state=opened&label_name%5B%5D=ai-framework%3A%3Aready%20for%20refinement&first_page_size=100).

For each issue, each team member will review the issues and provide the

following information:

* Estimated weight.

* How to break down the issue into different issues or merge requests.

Some considerations:

* Keep the conversation on the original issues.

* During this process, the issue description and labels should be updated as

more information is gathered.

* For efficiency, engineers can also skip the refinement of some issues

depending on the feedback that we already have, provided the issue still meets our definition of ready.

* Where the fix is clear and easy, we can assign the issue to

ourselves, give it a weight of 1 and push the fix.

An issue is more likely to be scheduled for development if it meets our definition of ready.

## Weighting and Estimation Process

### Weight Guidelines

Issues are weighted using the Fibonacci sequence (0, 1, 2, 3, 5, 8, 13+):

* **Weight 0:** Smallest issues (typos, minor formatting, simple code changes without tests)

* **Weight 1:** Simple issues with minimal uncertainty (good for new contributors)

* **Weight 2:** Straightforward issues requiring multiple code/test updates

* **Weight 3:** Larger issues with some complexity but manageable scope

* **Weight 5:** Should typically be broken down; acceptable for large manual updates with low risk

* **Weight 8/13+:** Placeholder weights indicating need for breakdown; too large or uncertain for immediate implementation

### Weight Update Process

Every issue assigned to the upcoming milestone needs to be weighed before applying the Deliverable label by Engineering Manager. Engineering Manager needs to check whether weight is assigned and, in case of the weight being equal or above 5, works on breaking issues down into smaller ones.

Engineering manager and Product Manager are responsible for asking to weight issues assigned for the upcoming milestone during weekly team meetings. They should ask engineers to read issue descriptions before the meeting so they are ready to weight them and ask questions if needed. They can split this process between more than one meeting.

---

title: AI Core Infra:Model Evaluation Infra

description: "The Model Evaluation Infra functional team within AI Core Infra, focused on maintaining the Centralized Evaluation Framework for evaluating AI features at GitLab."

---

## Overview

The Model Evaluation Infra team is part of the AI Core Infra organization. The team owns the Centralized Evaluation Framework (CEF), which powers AI feature evaluation across GitLab.

This page is a work in progress.

## Key Information

| | |

|---|---|

| **Slack Channel** | `#f_model_eval_infra` |

| **Stage Label** | `devops::ai platform` |

| **Group Label** | `group::ai core infra` |

| **Category Labels** | `category:model evaluation infra` |

## Documentation

- [CEF developer documentation](https://gitlab.com/gitlab-org/modelops/ai-model-validation-and-research/ai-evaluation/prompt-library/)