Commit 27042c2b authored by Andy Hohenner's avatar Andy Hohenner
Browse files

Updating performance contracts with latest changes

parent 154fb2bd
Loading
Loading
Loading
Loading
+4 −4
Original line number Diff line number Diff line
@@ -126,7 +126,7 @@ flowchart LR

  subgraph RUNNER[CI Runner - Browser Performance Job]
    VALIDATE[Schema validation]
    MERGE[merge_budgets.py]
    MERGE[merge_budgets.js]
    SITESPEED[SiteSpeed run\n--budget.configPath merged.json]
  end

@@ -306,9 +306,9 @@ This design document describes the contract model and architecture decisions (CP
Summary of the frontend pilot (high-level):

- We pilot a developer-centric workflow for frontend performance budgets using SiteSpeed. Budgets and URL lists are colocated in the `sitespeed-measurement-setup` repository under `performance/` so developers can update them together in a single PR.
- Budget files are organized as environment defaults (`performance/budgets/environments/*.json`) plus optional per-team overrides (`performance/budgets/teams/*.json`). At CI runtime, the environment budget is merged with a team override; team entries override environment entries on collision (keyed by `(url,metric)`).
- Budget files are organized as a baseline default (`performance/budgets/default.json`), environment overrides (`performance/budgets/environments/*.json`), and optional per-team overrides (`performance/budgets/teams/*.json`). All files use SiteSpeed's native nested budget format. At CI runtime, the environment budget is merged with a team override; team entries override environment entries at the metric level within each section, and URL/alias-keyed overrides are merged independently from global section defaults.
- MR-level runs are advisory and run SiteSpeed locally with `--budget.configPath` against a review app URL. The initial pilot avoids submitting MR runs to the central sitespeed-runway server to prevent uncontrolled data growth; scheduled or protected-branch runs can be promoted later.
- The `sitespeed-measurement-setup` POC (branch `add-performance-contracts`) contains examples, a JSON Schema, and helper scripts (`validate_budget.py`, `merge_budgets.py`).
- The `sitespeed-measurement-setup` repo contains examples, a JSON Schema, and helper scripts (`validate_budget.js`, `merge_budgets.js`) under `performance/`.

Example `performance.yml` frontend config (namespaced):

@@ -326,6 +326,6 @@ frontend:
  default_budget: mr
```

Merge semantics note: the environment budget and optional team budget are merged at runtime; entries are keyed by `(url,metric)` and team entries override environment entries on collision.
Merge semantics note: the environment budget and optional team budget are merged at runtime using SiteSpeed's native nested format; team entries override environment entries at the metric level within each section, and URL/alias-keyed overrides are merged independently from global section defaults.

Refer to the handbook page for the SiteSpeed examples and CI snippets; this design doc focuses on architectural decisions and the CPT roadmap.
+8 −8
Original line number Diff line number Diff line
@@ -91,7 +91,7 @@ We support a lightweight, developer‑centric workflow for frontend performance

- Budgets live alongside the test URL lists in the `sitespeed-measurement-setup` repository under a `performance/` directory. This keeps URLs and budgets versioned together and makes local developer runs straightforward.
- The main repository contains a `performance.yml` entry that acts as the launching point for CI: it references the environment budget and optional per‑team budget files in the `sitespeed-measurement-setup` submodule.
- At CI runtime the chosen environment budget is merged with an optional per‑team budget. Merge semantics are deliberately simple: each budget entry is keyed by `(url, metric)`; team entries override environment entries on collision and otherwise entries are concatenated.
- At CI runtime the chosen environment budget is merged with an optional per‑team budget. Merge semantics are deliberately simple: budgets use SiteSpeed's native nested format; team entries override environment entries at the metric level within each section, and URL/alias-keyed overrides are merged independently from global section defaults.
- MR‑level SiteSpeed runs are advisory by default (the MR job runs SiteSpeed locally in the pipeline using the Browser‑Performance pattern and the merged budget via `--budget.configPath`). MR runs are `allow_failure: true` and are opt‑in via an MR label or manual trigger.

```mermaid
@@ -107,7 +107,7 @@ flowchart LR

  subgraph RUNNER[CI Runner - Browser Performance Job]
    VALIDATE[Schema validation]
    MERGE[merge_budgets.py]
    MERGE[merge_budgets.js]
    SITESPEED[SiteSpeed run\n--budget.configPath merged.json]
  end

@@ -135,15 +135,15 @@ performance/
  README.md
  schema/budget.schema.json
  budgets/
    default.json
    environments/{production,staging,mr}.json
    teams/{<team>.json}
  scripts/
    validate_budget.py
    merge_budgets.py
  requirements.txt
    validate_budget.js
    merge_budgets.js
```

The `validate_budget.py` performs JSON Schema validation. `merge_budgets.py` produces a merged budget JSON implementing the team-overrides-environment semantics. CI should run the validator on PRs that change budgets.
The `validate_budget.js` performs JSON Schema validation. `merge_budgets.js` produces a merged budget JSON implementing the team-overrides-environment semantics. CI should run the validator on PRs that change budgets.

Developer flow (summary):

@@ -206,9 +206,9 @@ frontend:

Notes on behavior:

- The CI runner merges the chosen environment budget and the optional per-team budget using a deterministic rule: team entries override environment entries on (url,metric) matches; otherwise entries are concatenated. The merged JSON is passed to SiteSpeed with `--budget.configPath`.
- The CI runner merges the chosen environment budget and the optional per-team budget using a deterministic rule: budgets use SiteSpeed's native nested format; team entries override environment entries at the metric level within each section. URL/alias-keyed overrides (e.g. `"CodeReview_MR_List": { ... }`) are merged independently from global section defaults. The merged JSON is passed to SiteSpeed with `--budget.configPath`.
- MR-level runs are advisory (`allow_failure: true`) and run SiteSpeed locally in the Browser-Performance job pattern against the review-app URL. We intentionally avoid submitting MR runs to the central sitespeed-runway runner in the initial pilot to prevent data flooding while budgets are tuned.
- The `sitespeed-measurement-setup` repo contains a POC branch `add-performance-contracts` with example budget files, a JSON Schema, and two helper scripts (`validate_budget.py`, `merge_budgets.py`). CI should run validation on budget file changes.
- The `sitespeed-measurement-setup` repo contains example budget files, a JSON Schema, and two helper scripts (`validate_budget.js`, `merge_budgets.js`) under `performance/`. CI should run validation on budget file changes.

Note: the schema accepts a `frontend` object. The presence of the object implies frontend contracts are configured; the optional `enabled` boolean can be used to explicitly opt-out or opt-in when needed.