Performance contracts for Modular Features
## Problem Statement
There is no single, machine-readable source of truth that defines what "good performance" means for a modular service and that can be validated automatically in CI. As a result:
- Performance regressions are caught late.
- Resource limits in Helm charts, Runway manifests, and Bench assemblies can drift out of sync with actual performance targets.
- AI coding agents generate code with no awareness of latency budgets, resource constraints, or other performance-impacting aspects.
- Teams lack a standardized, repeatable process for defining and enforcing performance requirements across the modular service lifecycle.
We propose the modular feature architecture include a performance contract layer — a declarative specification that binds latency budgets, error rates, resource limits, and SLI mappings into one artifact that travels with the service and is enforced automatically.
## Value Proposition
Performance contracts create a shift-left feedback loop for modular service performance. By encoding performance targets into a validated YAML file (`performance.yaml`) that drives CI gating, load testing, resource provisioning, and AI agent guidance, we achieve:
1. **Earlier detection of regressions** — every MR is validated against the contract.
2. **Single source of truth** — one file defines what Helm charts, Runway manifests, Bench assemblies, dashboards, and alerting rules should be enforcing.
3. **AI-aware performance governance** — a skill published to the [GitLab Skills repo](https://gitlab.com/gitlab-org/ai/skills) gives AI coding assistants concrete, machine-readable rules derived from the contract. Because guidance lives in one place it is accessible to agents across all modular feature repos and requires only a single update as the contract system evolves.
4. **Standardised adoption path** — a reusable contract schema and validation toolkit means any modular service can adopt performance contracts efficiently.
## Alignment to Q2 Performance Enablement Goals
| Q2 Goal | How it aligns |
|---------|---------------|
| Put performance in the agent, not in the developer's head | The contract file (`performance.yaml`) and its derivative GitLab Skill give AI coding assistants concrete, machine-readable rules. Developers no longer need to memorize or look up performance constraints — the agent is aware of them. |
| Automate the checks so developers don't have to run them | Contract validation and load tests execute automatically on every MR via CI. Developers push code and get a pass/fail verdict. |
| Define the standard so developers never have to | The reusable contract schema, scaffolding, and CI template provide a turnkey standard for any modular service. Teams adopt performance contracts by running one command and adding two lines to their `.gitlab-ci.yml`. |
## Proposed Solution
### Architecture
| Layer | Purpose |
|-------|---------|
| Contract definition (`performance.yaml`) | Single source of truth for performance expectations |
| Instrumentation | Metrics emission from service runtime |
| Validation | Schema linting and semantic checks in CI |
| Load testing | Load test execution against contract thresholds |
| Ecosystem alignment | Deployment configs, dashboards, and agent guidance derived from the contract |
The contract file (`performance.yaml`) is the entry point for all tracks. It defines performance expectations for a service — latency percentile targets, error rate thresholds, and resource budgets. For services using sitespeed for frontend metrics, the contract references the corresponding sitespeed budget file rather than embedding frontend thresholds directly. Validation, load testing, and ecosystem alignment are each handled by track-specific tooling that reads from this single source of truth.
### Scope Boundary
**Environment management is explicitly out of scope.** The performance contract system does not own the definition, provisioning, or teardown of test environments. The contract defines *what* to measure and *what thresholds to enforce*; the environment substrate is a dependency managed externally. CPT is the leading candidate for providing transitory MR-level environments and will be investigated as a dependency.
### Exploratory POC
[A working POC](https://gitlab.com/gl-dx/performance-enablement/demos/perf-contract-poc) has been built and validated end-to-end — from contract definition through to CI gating — using a dummy Go service. A [walkthrough](https://drive.google.com/file/d/1bz2IwUE80H0MspLT0-TiFj3poWaEa9Cc/view?usp=drive_link) is available with more detail.
### Three MVP Tracks
Milestone 2 is being delivered as three parallel MVP tracks, each targeting a different service type and test runner. Each track has its own child epic and pilot team.
| Track | Epic | Pilot | Schema | Environment | Status |
|-------|------|-------|--------|-------------|--------|
| **Frontend / sitespeed** | [&190 MVP: Sitespeed-backed performance contract — Rapid Diffs pilot](https://gitlab.com/groups/gitlab-org/quality/quality-engineering/-/work_items/190) | Rapid Diffs team | Sitespeed native budget format (separate from `performance.yaml`) | CNG child pipeline, label-gated (`pipeline:run-sitespeed`) | :construction_site: Phase 4 — CI wiring in progress |
| **Backend / k6** | [&191 MVP: Backend performance contract — k6/GPT pilot](https://gitlab.com/groups/gitlab-org/quality/quality-engineering/-/work_items/191) | TBD | `performance.yaml` (finalized in team-tasks#4406) | CPT (leading candidate) | :hourglass_flowing_sand: Phase 1 — tooling decision and pilot team identification pending |
| **OpenAPI** | [&192 MVP: OpenAPI-backed performance contract — API pilot](https://gitlab.com/groups/gitlab-org/quality/quality-engineering/-/work_items/192) | TBD (API team has soft interest) | `performance.yaml` with OpenAPI spec reference extension | TBD | :hourglass_flowing_sand: Phase 1 — investigation not yet started |
The sitespeed track uses a **separate schema** (sitespeed's native budget JSON format) rather than `performance.yaml`. This was a deliberate decision to keep the frontend MVP shippable without blocking the M2 timeline for the backend track or introducing a translation layer. Schema unification is deferred to M4+.
## Milestones
**Status:** :hourglass_flowing_sand: Not started · :construction_site: In progress · :white_check_mark: Done
### Milestone 1 — Prep
*Done when: contract schema is finalized, environment management approach is decided, and a pilot team candidate is identified.*
The goal of this milestone is to make the decisions that everything else depends on before any significant build work begins. No tooling is built until these gates are cleared.
| # | Task | Description | Status |
|---|------|-------------|--------|
| 1.1 | [Review POC `performance.yml` and finalize contract schema framework](https://gitlab.com/gitlab-org/quality/quality-engineering/team-tasks/-/work_items/4406) | Finalize the schema, define performance tiers as scaffolding defaults, and produce a contract authoring guide. | :white_check_mark: |
| 1.2 | [Evaluate and finalize environment management approach for MR-level contract runs](https://gitlab.com/gitlab-org/quality/quality-engineering/team-tasks/-/work_items/4407) | Evaluate CPT as the environment substrate and document the decision with rationale. | :white_check_mark: |
| 1.3 | [Identify and soft-engage pilot team candidate for Milestone 3](https://gitlab.com/gitlab-org/quality/quality-engineering/team-tasks/-/work_items/4408) | Soft-identify a modular feature team willing to participate in Milestone 3 for each track. Availability confirmed late is the primary scheduling risk for this epic. | :white_check_mark: Frontend (Rapid Diffs)<br><br>:construction_site: Backend (TBD)<br><br>:construction_site: OpenAPI (API team — soft interest, formal engagement pending) |
### Milestone 2 — MVP
*Done when: each track has a working advisory-mode CI contract gate producing a pass/fail verdict in CI.*
Milestone 2 is being delivered as three parallel tracks (see [Three MVP Tracks](#three-mvp-tracks) above). Progress below reflects the state of each track independently.
| # | Task | Description | Frontend track | Backend track | OpenAPI track |
|---|------|-------------|-----------------|---------------|---------------|
| 2.1 | Adapt POC to chosen environment tool | Port the contract definition → validation → enforcement → CI gating flow to the environment substrate for each track. | :construction_site: Phase 4 — CNG child pipeline wiring ([team-tasks#4449](https://gitlab.com/gitlab-org/quality/quality-engineering/team-tasks/-/work_items/4449), draft MR [gitlab!244227](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/244227)) | :hourglass_flowing_sand: Tooling decision (CPT vs GPT) pending | :hourglass_flowing_sand: Not started |
| 2.2 | Structured CI reporting | Make contract validation and test results consumable by the GitLab MR widget so pass/fail verdicts appear inline. Initial failure mode is **non-blocking (advisory)**. | :construction_site: In progress — budget pass/fail exit code propagation being confirmed | :hourglass_flowing_sand: Not started | :hourglass_flowing_sand: Not started |
| 2.3 | Publish reusable CI/CD template | Create a shareable CI configuration that any modular service can include in a couple of lines to get contract validation and smoke testing with sensible defaults. | :hourglass_flowing_sand: Not started | :hourglass_flowing_sand: Not started | :hourglass_flowing_sand: Not started |
| 2.4 | [Integrate performance contract scaffolding into common-template-copier](https://gitlab.com/gitlab-org/quality/component-performance-testing/-/work_items/146) | Extend the common-template-copier so that new modular service repos are created with a starter performance contract and corresponding CI jobs. | :hourglass_flowing_sand: Not started | :hourglass_flowing_sand: Not started | :hourglass_flowing_sand: Not started |
### Milestone 3 — Pilot Adoption
*Done when: the pilot modular service has an authored performance contract, passing in CI, with schema refinements addressed.*
Runs in parallel with Milestone 4. Active pilot team engagement begins once Milestone 2 is stable.
| # | Task | Description | Status |
|---|------|-------------|--------|
| 3.1 | Activate pilot team | Formally engage the team identified in Milestone 1. Walk through the contract authoring process, scaffold their `performance.yaml` using Milestone 2 tooling, and support them through initial CI integration. | :hourglass_flowing_sand: |
| 3.2 | Author and validate pilot contract | Work with the pilot team to define their contract against real service data. Tune thresholds until CI is green. | :hourglass_flowing_sand: |
| 3.3 | Address schema gaps | Capture friction and schema gaps discovered during pilot adoption. Address within Milestone 3 or explicitly defer to Milestone 5. | :hourglass_flowing_sand: |
| 3.4 | Collect structured feedback | Document the pilot team's experience with contract authoring, CI integration, and threshold calibration. Use as input to refine the adoption playbook. | :hourglass_flowing_sand: |
### Milestone 4 — AI Integration
*Done when: agents have access to performance contract guidance and contract tests are running in the AI-assisted workflow.*
Runs in parallel with Milestone 3. 4.1 can begin as soon as the schema is stable at the end of Milestone 1.
| # | Task | Description | Status |
|---|------|-------------|--------|
| 4.1 | Publish performance contract skill to GitLab Skills repo | Author and publish a skill to the [GitLab Skills repo](https://gitlab.com/gitlab-org/ai/skills) covering: contract format and schema, how tests are run and when to apply them, and links to functional contract testing so agents have a complete structural + performance picture. Content is informed by Milestone 1 schema work and refined post-pilot. | :hourglass_flowing_sand: |
| 4.2a | Get contract tests running in CPT AI workflow | Integrate contract test execution into the CPT AI workflow. Pass/fail verdict is surfaced without analysis — this is a complete, standalone deliverable that adds value independent of the analysis step. | :hourglass_flowing_sand: |
| 4.2b | AI analysis of contract results posted to MR | Surface AI-generated analysis of contract test results as MR feedback. Expected to require iteration due to performance contracts measuring different metrics than standard CPT tests. Nice-to-have for Q2; moves to Milestone 5 if capacity is not available. | :hourglass_flowing_sand: |
### Milestone 5 — Q3 Backlog
A holding area for items discovered during Q2 that are deferred for the next iteration. Scope will be formalised into Q3 planning when ready.
| Item | Notes |
|------|-------|
| Contract staleness detection | Scheduled checks that detect contracts not reviewed within a defined period and auto-open issues for the owning team. |
| 4.2b AI analysis (if deferred) | See Milestone 4 above. |
| Additional service adoption | Roll out to 2–3 additional modular services beyond the Milestone 3 pilot. |
| Schema versioning and extension | Breaking vs non-breaking change strategy, extension points for service-specific fields. |
| Prometheus/Grafana Integration | Evolve from per-run CI artifacts (MVP) to persistent Prometheus/Grafana dashboards, enabling long-range performance trend analysis across MRs and releases. |
## Success Criteria
| Criterion | Target | Measurement |
|-----------|--------|-------------|
| MR-level regression detection rate | ≥ 80% of performance regressions caught before merge | Track regressions found by contract CI jobs vs. regressions found over a 3-month window post-MVP. |
| Adoption breadth | ≥ 3 Modular services with active contracts | No. of repos with a valid `performance.yaml` passing CI. |
| Time to adopt | < 1 day for a new service using scaffolding CLI + CI template | Measured from "developer starts" to "first green MR pipeline with contract validation." |
| Developer satisfaction | Net positive feedback from adopting teams | Qualitative survey of pilot and early adopter teams. |
---
## Dependencies
| Dependency | Description | Risk |
|------------|-------------|------|
| **LabKit v2 stable release** | LabKit v2 is in active development; the POC uses a local `replace` directive. | Medium — API changes could require contract schema updates. |
| **Pilot team availability** | Milestone 3 requires a willing pilot team to partner with. | Medium — team bandwidth and prioritization may delay adoption. |
| **Bench / Runway alignment** | Deployment config alignment requires coordination with the Bench and Runway teams. | Medium — real integration requires coordination beyond what the POC covers. |
| **Environment management / CPT** | CPT is the leading candidate for providing transitory MR-level test environments. | High — if CPT cannot serve this role, an alternative must be identified before Milestone 2 build work begins. |
---
## Open Questions
1. Who owns the contract schema long-term? Should it live in a shared `gitlab-org` repo or be managed by the Performance Enablement team? What is the review process for schema changes?
2. How do teams determine initial p95/p99 targets for a new service with no production data? Should we provide tier-based defaults (e.g., "fast read = 100ms p95" as a starting point)?
3. How do performance contracts relate to GitLab's existing error budget and SLO tooling? Should contract thresholds be derived from SLOs, or should SLOs be derived from contracts?
<!-- STATUS NOTE START -->
## Status 2026-07-30
We're building automated performance contracts that catch regressions before merge and give teams a discoverable single source of truth for what 'good performance' means for any service.
:clock1: **total hours spent this week by all contributors**: 10
Progress this week was slow due to R&D Summit
:tada: **achievements**:
- Sitespeed performance contracts (called budgets) are advancing on multiple fronts
- Working on [deploying the CI trigger](gitlab-org/quality/sitespeed-runway#35)
- Turned on CI Pipeline, troubleshooting and monitoring to verify functionality
- [Sitespeed on MR](gitlab-org/quality/quality-engineering/team-tasks#4449): Continuing to work on getting sitespeed running in MR pipelines (see blocker below)
:issue-blocked: **blockers**:
- there is an unrelated failure on Hugolint for the main GitLab repo that is causing the Sitespeed on MR pipeline to fail before the tests are run gitlab-org/gitlab!247949+
:arrow_forward: **next**:
- [Define the budget metrics](gitlab-org/quality/quality-engineering/team-tasks#4448)
- Expand Sitespeed in MRs capabilities [to run more tests](gitlab-org/quality/quality-engineering/team-tasks#4581)
- [Push sitespeed budget data to Grafana](gitlab-org/quality/sitespeed-runway#31)
- [Pilot team](gitlab-org/quality/quality-engineering/team-tasks#4408) status: frontend track has an active pilot; backend track not yet started; OpenAPI track has soft interest from the API team, formal engagement pending
- Start investigating OpenAPI Performance Contracts integration (on radar, not yet started)
_Copied from https://gitlab.com/groups/gitlab-org/quality/-/epics/387#note_3625833635_
<!-- STATUS NOTE END -->
epic