## Problem Today, GitLab's code review workflows treat all MRs with the same level of scrutiny by default. Teams that want to differentiate — routing high-risk changes to more senior reviewers and letting low-risk changes merge quickly — must configure this manually through CODEOWNERS and branch protection rules. These approaches share a critical flaw: **they match on file paths, not on semantic risk**. A safe refactor touching 50 files gets the same review requirements as a 3-line change to an authentication critical path. This creates two compounding problems: - **Low-risk MRs sit in queues**, blocking engineers who need to ship while reviewers are occupied - **High-risk MRs don't get enough attention**, because the same queue pressure means reviewers can't give them the depth they deserve As AI-assisted development accelerates code production, this problem gets worse. Teams are creating more MRs than humans can reliably review, and the inability to triage by actual risk is a significant bottleneck to both velocity and quality. Existing AI code review tools address pieces of this problem, but none closes the loop into tiered AI-driven review routing. ## Proposal Introduce **MR risk classification** to Duo Code Review: an agent-driven system that evaluates each MR when it is opened and assigns it a risk tier. Classification runs **once**, in **parallel with base Duo Code Review**, at MR open or ready-for-review time. It does not re-run on re-review requests and does not use the Duo Code Review output as input. The tier then drives an automated recommendation for what level of review is appropriate. Proposed tiers: | Tier | Risk Level | Recommended Action | |------|------------|--------------------| | 1 | Very Low / Low | Auto-approve or fast-track merge | | 2 | Medium | Standard Duo Code Review | | 3 | High | Recommend Advanced Code Review | The tier assignment should be **explainable** — every classification surfaces a plain-language rationale (e.g., _"This MR is Tier 3 because it modifies `/ee/lib/auth/session_token.rb` and affects 4 downstream API consumers"_) so reviewers can immediately act on or challenge it. ## Risk Classification Signals The classifier should weight signals including but not limited to: **Semantic signals (highest weight)** - Changes to authentication, authorization, session, or encryption logic - API contract or schema modifications (breaking change detection) - Blast radius: number of services/consumers affected by the change, using Orbit's CALLS/DEFINES graph when available and path-pattern heuristics as a fallback - Behavioral vs. cosmetic change detection (code that alters runtime behavior vs. formatting/docs) - Change dispersion — whether changed files are logically related or scattered across unrelated subsystems **Structural signals** - Files changed are in security-sensitive directories (e.g., `ee/`, `config/`, `.gitlab-ci.yml`) - Cross-repo or cross-service impact detected - Changes to shared libraries or platform-level code **History-based signals (GitLab-native advantage)** - SAST/DAST findings in the affected code paths - Pipeline failure rate on the affected files - Deployment frequency and incident history for the affected area - Prior review patterns: how similar MRs have been reviewed in the past - Author commit history on changed files (author familiarity with the code paths being modified) - Existing test coverage on changed lines, not only whether test files were modified alongside the change **Code generation signals** - MR was authored primarily by an AI coding assistant, detected via service account author patterns (industry research shows AI-generated code has significantly higher defect rates than human-written code) - Future: per-source track record — weight by the incident/revert history of the specific AI tool or workflow that produced the MR, rather than treating all AI-generated code identically ## Domain Gates Certain domains carry enough inherent risk that they should establish a **minimum tier floor regardless of rubric score**. Domain gates are a pre-scoring categorical override layer — they are not inputs to the weighted rubric, they override it. When a domain gate fires, it emits a `domain_tag` and a `minimum_tier_floor`. The final tier is `max(rubric_tier, domain_floor)`. Multiple gates can fire on a single MR. **Platform default gates:** | Domain | Detection | Minimum Tier | |--------|-----------|-------------| | Auth / session / authorization | Semantic + path patterns | HIGH | | Database migration | `db/migrate/`, `.sql`, migration class patterns | HIGH | | Cryptography / secrets | `cipher`, `encrypt`, `kms`, `vault`, secret management patterns | CRITICAL | | Payment processing | Semantic patterns (`stripe`, `billing`, `charge`, PCI-scope) | CRITICAL | | Compliance / audit / PII | `audit_event`, data retention, PII handling patterns | HIGH | | Infrastructure / deployment | `Dockerfile`, `*.tf`, `k8s`, `.gitlab-ci.yml` | HIGH | **Configuration:** GitLab ships platform defaults. Groups can override thresholds or add domains at group level. Projects can register custom domain gates using CODEOWNERS-style rules — path patterns plus semantic keywords, with a declared minimum tier. **Routing output:** `domain_tags` produced by this step flow downstream to [Advanced Duo Code Review / Specialized Agents (21524)](https://gitlab.com/groups/gitlab-org/-/work_items/21524), where they determine which specialized agents (security, database, etc.) are recommended for the MR. ## Key Behaviors - **Runs once**: Classification fires at MR open or ready-for-review time, in parallel with base Duo Code Review. It does not re-run on re-review requests, new commits, or draft state changes. - **Staleness indicator**: When new commits are pushed after classification, the existing classification note shows which diff version was assessed and how many commits have been added since (e.g., _"Risk classified against diff v3 — 2 commits added since"_). Re-evaluation on update is explicitly out of scope. - **Break-glass override**: Teams can override a tier classification with a required written justification. Overrides are logged for audit and used to calibrate the model over time. - **Confidence surfacing**: The tier assignment includes a confidence level. A borderline Tier 2/3 classification is surfaced differently from a clear-cut one, giving reviewers useful signal about uncertainty. - **Configurable thresholds**: Teams can tune what constitutes High risk for their codebase — a fintech team may want to flag currency logic as Tier 3; a docs team may never trigger Tier 3 at all. Custom domain gates (see above) are the primary mechanism for this. ## Why This Is a GitLab Differentiator External AI code review tools are largely constrained to working from the diff and repository graph alone. Duo has access to signals no standalone tool can replicate: - **GitLab CI/CD history** on affected code paths (pipeline failure rates, flaky tests) - **SAST/DAST findings** surfaced during prior MRs to the same files - **Deployment frequency and incident data** for the affected service areas - **Issue and milestone context** — what was the intent behind this change? - **Review history** — who has approved similar changes before, and did those changes cause incidents? - **Orbit knowledge graph** — cross-file call and dependency relationships for accurate blast radius scoring Leading context engines in this space achieve around 80% classification accuracy using codebase context alone. Adding GitLab's platform signals to that foundation should significantly exceed what any standalone tool can achieve, and creates a durable competitive moat. ## Open Questions - What is the right default tier threshold for auto-approve behavior, and should it be opt-in at the group/project level? - How do we handle the cold-start problem for new projects with no review history? - Should the break-glass override be a GitLab UI action (button on the MR), a slash command, or both? - How do we expose tier calibration data to platform admins so they can see where the model is wrong? - Should the risk tier be visible to the MR author, or only to reviewers? - Should domain tags be surfaced in the MR UI (similar to labels), or only through the internal classification note and 21524 routing? - At what point does Orbit indexing become a prerequisite rather than optional enrichment for blast radius scoring, and what is the accuracy difference when falling back to path-pattern heuristics?