Commit 33a214cc authored by Saurabh Chopra's avatar Saurabh Chopra
Browse files

Add GitLab Brain as a new handbook section 

Add /handbook/engineering/gitlab-brain/ with its own charter: Event
Platform (linking the existing Events Instrumentation page, plus
Ingestion and Streaming and Data Schema, Ontology & Contracts), Data
Infrastructure (Data Storage; Message Queue and Replication/CDC), Code
Search (Global Code Search Experience; Search Indexes and
Infrastructure), and Intelligence (Query and Semantics; Data
Intelligence Agent; Software Intelligence UI).

Purely additive: no existing page, link, image, or redirect changes.
Where a team page already exists (Events Instrumentation, the planning
pages, Database Excellence, Global Search), the new section links to it
in place. The only change outside the new directory is one CODEOWNERS
line for the new section.

Co-Authored-By: default avatarClaude Fable 5 <noreply@anthropic.com>
parent 038941be

.gitlab/CODEOWNERS

+1 −0
Changes for .gitlab/CODEOWNERS: 1 added line, 0 removed lines.
Original line number Diff line number Diff line
@@ -119,6 +119,7 @@ 
/content/handbook/engineering/data-engineering/database-excellence/database-automation/ @dazhu1

/content/handbook/engineering/data-engineering/database-excellence/database-health/ @alexives

/content/handbook/engineering/data-engineering/database-excellence/self-managed-database-experience/ @nbelokolodov

/content/handbook/engineering/gitlab-brain/ @nitinsinghal74 @chopras



^[Eng-AI-department]

/content/handbook/engineering/ai/ @timzallmann @mnohr @bastirehm @mcorren @dashaadu @donaldcook @zhaider5 @aelhusseiny @wortschi @aying-gl @samdbeckham

content/handbook/engineering/gitlab-brain/_index.md

0 → 100644
+57 −0
Changes for content/handbook/engineering/gitlab-brain/_index.md: 57 added lines, 0 removed lines.
Original line number Diff line number Diff line 
---

title: "GitLab Brain"

description: "In the agent era, GitLab's advantage is what it knows: the code, the reviews, the pipelines, and the permission model that governs them. GitLab Brain turns that knowledge into a working mind — observed as it happens (events), remembered in durable infrastructure (storage), recalled instantly (search), reasoned over into insight. This is how GitLab becomes a system of intelligence."

---



## Mission



We power GitLab's intelligence — for every person and every agent who builds.



## Organization Structure



Each group is one function of the brain: the Event Platform observes, Data Infrastructure processes and remembers, Code Search recalls, and Intelligence reasons.



```mermaid

flowchart LR

    BRAIN[GitLab Brain]

    click BRAIN "/handbook/engineering/gitlab-brain/"



    BRAIN --> EP[Event Platform]

    click EP "/handbook/engineering/gitlab-brain/event-platform/"

    BRAIN --> DI[Data Infrastructure]

    click DI "/handbook/engineering/gitlab-brain/data-infrastructure/"

    BRAIN --> SR[Code Search]

    click SR "/handbook/engineering/gitlab-brain/search/"

    BRAIN --> INT[Intelligence]

    click INT "/handbook/engineering/gitlab-brain/intelligence/"



    EP --> EI[Events Instrumentation SDKs]

    click EI "/handbook/engineering/data-engineering/analytics/analytics-instrumentation/"

    EP --> IS[Ingestion and Streaming]

    click IS "/handbook/engineering/gitlab-brain/event-platform/ingestion-and-streaming/"

    EP --> DC[Data Schema, Ontology & Contracts]

    click DC "/handbook/engineering/gitlab-brain/event-platform/data-contracts/"



    DI --> DS[Data Storage]

    click DS "/handbook/engineering/gitlab-brain/data-infrastructure/data-storage/"

    DI --> MQ[Message Queue and Replication/CDC]

    click MQ "/handbook/engineering/gitlab-brain/data-infrastructure/message-queue-and-replication/"



    SR --> GSE["Global Code Search Experience (Retrieval and Ranking)"]

    click GSE "/handbook/engineering/gitlab-brain/search/experience/"

    SR --> SI[Search Indexes and Infrastructure]

    click SI "/handbook/engineering/gitlab-brain/search/indexes/"



    INT --> QS[Query and Semantics]

    click QS "/handbook/engineering/gitlab-brain/intelligence/query-and-semantics/"

    INT --> DIA[Data Intelligence Agent]

    click DIA "/handbook/engineering/gitlab-brain/intelligence/data-intelligence-agent/"

    INT --> SIU[Software Intelligence UI]

    click SIU "/handbook/engineering/gitlab-brain/intelligence/software-intelligence-ui/"

```



## How we work



- [Quarterly planning](/handbook/engineering/data-engineering/quarterly-planning/): how we plan a quarter: ranked, right-sized bets inside a capacity ceiling, a 20-minute review every fourteen days, swap and deprioritize rules, and a say/do score at quarter close.

- [Priorities tracker guide](/handbook/engineering/data-engineering/priorities-tracker/): the companion reference for the tracker spreadsheet: the tabs, the Two-pager, the columns on a row, the capacity arithmetic and the checks a row must pass before plan lock.

- [Pre-mortems for launches](/handbook/engineering/data-engineering/pre-mortems/): an optional, forty-five-minute practice for surfacing launch-specific risks before the ship date. The launch DRI decides whether to run one.

content/handbook/engineering/gitlab-brain/data-infrastructure/_index.md

0 → 100644
+13 −0
Changes for content/handbook/engineering/gitlab-brain/data-infrastructure/_index.md: 13 added lines, 0 removed lines.
Original line number Diff line number Diff line 
---

title: "Data Infrastructure"

description: "Runs GitLab's datastores and data movement as internal services. Every system here has an owner who answers for its SLOs, capacity, cost, and upgrades."

---



## Scope



We run GitLab's datastores and data movement as internal services. Every system here has an owner who answers for its SLOs, capacity, cost, and upgrades. We don't write product queries or business logic: the [Event Platform](/handbook/engineering/gitlab-brain/event-platform/) writes into this layer, and [Intelligence](/handbook/engineering/gitlab-brain/intelligence/) reads from it.



## Teams



- [Data Storage](data-storage/): ClickHouse and GitLab's analytics datastores.

- [Message Queue and Replication/CDC](message-queue-and-replication/): NATS and Siphon, the messaging backbone and change data capture.

content/handbook/engineering/gitlab-brain/data-infrastructure/data-storage/_index.md

0 → 100644
+8 −0
Changes for content/handbook/engineering/gitlab-brain/data-infrastructure/data-storage/_index.md: 8 added lines, 0 removed lines.
Original line number Diff line number Diff line 
---

title: "Data Storage"

description: "Runs ClickHouse and GitLab's analytics datastores as internal services."

---



## Scope



This team runs [ClickHouse](/handbook/company/working-groups/clickhouse-datastore/) and the rest of GitLab's analytics datastores. It is a complicated-subsystem team: deep expertise, narrow surface. The team answers for SLOs, capacity, cost, and upgrades, and stays out of product queries and business logic. Its first customer is [Query and Semantics](/handbook/engineering/gitlab-brain/intelligence/query-and-semantics/).

content/handbook/engineering/gitlab-brain/data-infrastructure/message-queue-and-replication.md

0 → 100644
+8 −0
Changes for content/handbook/engineering/gitlab-brain/data-infrastructure/message-queue-and-replication.md: 8 added lines, 0 removed lines.
Original line number Diff line number Diff line 
---

title: "Message Queue and Replication/CDC"

description: "Runs the pipes: NATS as the messaging backbone, and Siphon for replication and change data capture into the analytics stores."

---



## Scope



This team runs the pipes. NATS is the messaging backbone; [Siphon](https://gitlab.com/gitlab-org/analytics-section/siphon) handles replication and change data capture out of GitLab's [transactional databases](/handbook/engineering/data-engineering/database-excellence/) into [Data Storage](data-storage/). Producers write through the [Event Platform](/handbook/engineering/gitlab-brain/event-platform/) ingesters, and everything downstream reads from here against one published lag.