feat: Assert the converged Renovate idiom, and stop failing the opt-out

Standard

renovate.

Change type

  • Modify existing standard's check.sh (add, remove, or change an assertion)
  • Modify existing standard (prose only)
  • Modify existing standard's templates

What does this MR do and why?

The standard checked two things: .extends[0] names the catalog preset, and there is no gitlab block. That is a 2026-05-era invariant, and it has two problems.

It fails the estate's own opt-out. Read through jq -er, .extends[0] returns null on a config with no extends array, so every Renovate-excluded repository reports FAIL: jq could not read .extends[0]. Measured today across the 58 non-archived projects in the group: 35 ship a renovate.json, and 13 of those are the excluded class. The standard was manufacturing a false failure on 37% of its own subject population, against a marker its own SKILL.md documents as the way to opt out.

And it catches none of the drift it exists to catch, because nothing past extends[0] is read at all.

Three assertions now: extends-or-excluded, no key the preset owns, no top-level key outside the documented per-repo set.

Failure mode this addresses

Estate policy forked into files nobody re-reads when the preset moves.

Two instances, both live today. Three projects (storyping, p1-vp-vexer, p1-vp-generator) each carry a hand-written packageRule disabling sigstore/cosign in near-identical wording, which is one fact held three times and stale on each independently. Four projects carry "labels": ["maintenance", "renovate"] against the preset's ["renovate", "dependencies"], so the estate's label taxonomy does not agree with itself and a fleet query on either label misses part of the fleet.

The automerge case is the one that matters most, and it is why the second assertion reaches inside packageRules rather than only checking the top level. The customer-shipping floor is enforced by ordering: a later packageRule wins, and every rule naming a shipping class sits below the rules that grant automerge. A consumer's local automerge is evaluated after the whole preset, so it outranks the floor, and no reviewer of either file alone can see it.

Assertion deltas

- MODIFY: json-key:renovate.json:.extends[0]
       -> json-key:renovate.json:if .enabled == false then "excluded" else .extends[0] // "no-extends" end
- MODIFY: file-lacks:renovate.json:"gitlab"
       -> file-lacks:renovate.json:"(gitlab|schedule|labels|assignees|reviewers|automerge|
          platformAutomerge|minimumReleaseAge|internalChecksFilter|osvVulnerabilityAlerts|
          rebaseWhen|prConcurrentLimit|prHourlyLimit|prPriority|regexManagers)"
- ADD:    json-key:renovate.json:[keys[]] - [the eight per-repo keys] | ... | "bare"

Assertion 1 asks one question with two right answers. Either the extends array's first entry names the estate preset, or the file declares the repository Renovate-excluded with top-level enabled: false and a stated reason. An excluded file has no extends and should not have one: extending a preset whose entire content is suppressed is decoration. The // "no-extends" fallback is what turns a missing array into a reported value rather than a jq read error, so the failure names what is wrong instead of naming jq.

The parked class (dependencyDashboardApproval: true with a dated reason, for a repository that wants renovation but is blocked on something fixable) is admitted by assertion 3 alongside it. Park and exclude look alike from outside (both are dashboard-only and MR-silent) and mean opposite things, so the SKILL.md now states the distinction.

Assertion 2 is derived, not chosen. gitlab, schedule, assignees and reviewers are the four this standard's own SKILL.md has listed under "What does not belong" since it was written, and only the first was ever asserted. The rest are the top-level keys of the preset itself (renovate-config.json in the catalog), plus the two levers its own packageRule commentary calls load-bearing: prPriority, which is how the catalog bump outranks routine updates, and the concurrency limits, which the preset sets explicitly in two places because overriding one does not inherit the other. regexManagers is the deprecated spelling of customManagers, and the catalog ships the shared ones.

"gitlab" requires the closing quote, so the catalog repository's own "gitlabci" key does not match this and is judged by assertion 3 instead.

Assertion 3 is bare-extends purity. Eight top-level keys, each with a source that documents it: $schema, extends, customManagers (the campaign's convergence target is stated as "a bare extends plus only genuinely repo-specific custom managers"), packageRules (this standard's own SKILL.md lists three legitimate per-project cases, and assertion 2 is what stops them carrying preset policy), enabled, dependencyDashboardApproval, and description / // for the reason a class marker carries. The failure line names the offending keys, so it is the remediation list.

Deliberately not allowed, though one project uses them: gitlabci and dockerfile manager-scoping blocks. The catalog repository sets gitlabci.managerFilePatterns to the same three globs its own preset's catalog-pin custom manager already lists, and dockerfile.managerFilePatterns to the same glob the preset's Containerfile-ARG manager already lists. That is the preset and a consumer holding one fact twice.

A top-level key that appears in both assertion 2's list and outside assertion 3's allowlist is named twice. That is the cost of covering both positions: a line regex is the only thing that reaches inside a packageRule, and an allowlist is the only thing that catches a key nobody thought to enumerate. Both lines name the same remediation.

Consumer impact

Run against all 35 renovate.json files in the group, fetched from their default branches.

before after
pass 22 29
fail 13 (all false: the excluded class) 6 (all real)

The 6 failures:

Project Assertion What it carries
storyping 2 and 3 "labels": ["maintenance", "renovate"]
p1-vp-generator 2 and 3 same
p1-vp-generator-automation 2 and 3 same
p1-vp-vexer 2 and 3 same, plus postUpdateOptions
kaniko 2 automerge and labels inside a packageRule
pipeline 3 top-level gitlabci and dockerfile

All 13 excluded repositories now pass, as do reference, manifold (a packageRules entry that only groups, which is the documented legitimate case), and serviceping-redact (the deprecated pipeline//presets/renovate.json shim, which still resolves and is tolerated).

kaniko's automerge: false is worth naming, because it is not dangerous, it is redundant. The preset grants automerge only to lockFileMaintenance, pin, digest, and checksum-verified container tool pins under containers/**. chainguard-forks/kaniko matches none of those, so it already never automerges, and the local rule restates a property the preset guarantees. That is the shape the convergence is removing.

None of the 6 is remediated here. Each is an MR against the project, and for the label set the right MR is probably against gitlab-com/public-sector/pipeline: if maintenance is the label four projects want, it belongs in the preset once.

Enforcement: what runs today, and what fleet enforcement would take

Investigated and reported rather than implemented, because implementing it needs a decision this MR should not make.

What runs today. Nothing runs this standard against a consumer repository on any schedule or in any consumer pipeline. Concretely:

  • just check REPO=<path> from a reference checkout, invoked by a human or an agent. Nothing schedules it.
  • The reference's own self-conformance job runs just check against the reference only, with allow_failure: true.
  • There is no reference-check catalog component. That was decided on 2026-08-07 and the draft was closed unbuilt; the decision is recorded in README.md, in AGENTS.md, and cited in the catalog's own renovate-fleet-audit.py.
  • I read .gitlab-ci.yml from the default branch of ten projects including every flagship (tach, assay, manifold, postern, kaniko, egress, pipeline, reference, storyping, crucible). Zero run any reference conformance job. The two that grep positive for "conformance" are comments explaining that conformance runs locally, not jobs.
  • What does run fleet-wide for Renovate is the catalog's renovate-fleet-audit job, on scheduled and web pipelines, allow_failure: false. It audits delivery (each enrolled project's Dependency Dashboard for a write-side "Repository Problems" entry) and deliberately not configuration. The unasserted: field's claim that it exists is accurate; I verified the job and the script.

So the assertions in this MR are real and can fail, and today they fail only where someone points them. That is the honest state and it is stated in the SKILL.md.

What fleet enforcement would take. Three shapes, in increasing cost:

  1. Extend renovate-fleet-audit to read each enrolled project's renovate.json and apply these assertions alongside the delivery check. Cheapest by a wide margin: that job already enumerates the enrolled set, already holds a token with the right access, already classifies guest forks and labs, and already runs on a schedule. Roughly one function and a report section. It is a change to gitlab-com/public-sector/pipeline, so it is not in this MR and is proposed as follow-up.
  2. A scheduled fleet job inside the reference that shallow-clones every project in the group and runs just check REPO= against each. Entirely inside this repository, but it is a new mechanism with a token, a clone loop, and a blocking policy to decide, and it re-treads the ground the 2026-08-07 decision covered. Not small enough to land unasked.
  3. A conformance job per consumer, two lines each per the README recipe. 58 MRs, and it distributes the reference version pin across the fleet.

I did not implement any of them. Option 1 is the recommendation and belongs in an issue against the catalog.

Also found, not fixed

templates/lab/renovate.json, templates/docs/renovate.json and templates/paper/renovate.json each carry an identical inline customManagers entry bumping catalog pins. The preset already ships an equivalent manager covering .gitlab-ci.yml in any consumer, and the two disagree: the starters use gitlab-tags with depNameTemplate, the preset uses gitlab-releases with a hardcoded packageNameTemplate and a long comment explaining why. The starters pass the new assertions (customManagers is a legitimate per-repo key), so this is not a conformance failure, but three copies of a manager the preset already provides is the duplication the convergence exists to remove. Left for a separate MR so this one stays one concern.

Cutover schedule

Not applicable. The standard is already enforcing at v1.0.0 and stays there. This tightens what it asserts, and it is net less punitive: 13 false failures removed, 6 real ones added.

Test plan

  • just guard passes: guard: 11 primitives, checked 20 file(s), 0 violation(s). Only closed-set primitives.
  • PIPELINE_REPO=... just validate green, all seven gates
  • just check passes against this repo: the reference's own renovate.json is the bare form and satisfies all three
  • just ship green end to end
  • just sync run; the ## Verify block is generated from check.sh
  • Ran against all 35 consumer renovate.json files in the group, before and after, and the table above is the measured result. Verified able to fail on each new assertion and able to pass on the excluded class.
  • All four shipped templates (standards/renovate/templates, templates/{lab,docs,paper}) checked against the new assertions and pass

Reviewer checklist

  • check.sh sources ../../_lib.sh and calls report
  • SKILL.md front matter valid YAML; description and unasserted: both updated to match what is now asserted
  • "What this standard does not cover" corrected: it said MR review burden is "a per-project policy in packageRules, not a standard". It is an estate policy in the preset, and treating it as per-project is what let the overrides accumulate.
  • templates/renovate.json is the bare canonical form

References

Motivating research: the Renovate translation audit of 2026-08-11, section 1. Operator approved this change on 2026-08-11. Related: the operator's 2026-08-11 ruling moving ironbank to lab/ironbank as a Renovate-excluded repository, which is the class assertion 1 now admits.

Edited by Andrew Dunn

Merge request reports

Loading
Loading