chore(skills): teach write-readable-go the comment-cap ratchet
What changed
.claude/skills/write-readable-go/SKILL.md and docs/dev/go-style.md gain the
decision procedure for an edit that lands inside a comment block already over
its cap, plus a count bound on the banner carve-out.
The caps gate charges every line of a block once a diff touches any line in it, a pure deletion included, so correcting one sentence inside a 40-line legacy block reports the whole block. Neither artifact said what to do at that point. Both also described the banner carve-out in the singular — "a section banner costs the declaration under it nothing" — without bounding how many you may write, so converting a paragraph into a run of banners was the cheapest compliant edit available.
The five outcomes are all established practice: compress to the cap, relocate to
a site with budget, move a cross-package census to docs/dev/ behind a pointer,
restore the file from the merge base, or decline the edit and disclose it. The
skill carries them as an ordered list; go-style.md carries the same five as a
table with the trap on each, so the two do not drift.
The banner bound is deliberately not one of those five, and both artifacts
now say so. Its trigger is finding a run, not the gate reporting one, because
the gate never will: every gap-separated segment keeps its own row, so a run of
ten passes in silence. Both artifacts also now say where a run's claims go —
each onto the declaration it describes, which for a file header is usually a
declaration in another file, or into docs/dev/ behind a pointer when a claim
spans the package. That last part is what makes the bound actionable rather than
advisory; see Pass rate for what it did and did not move.
Four smaller gaps close alongside:
- A test's own strings — an assertion message,
t.Skiport.Fatalftext, a per-casewhycolumn — are a free slot neither artifact mentioned, and they surface on failure where a comment does not. - A mid-body block was listed among the free slots and is not one: it takes the 2-line cap of any other block, so a 3-line why relocated there is refused a second time. The body head is not a site at all, since it shares the doc comment's cap.
- A file header needs no banner: a detached block before the
importlist is attached to no declaration and already takes its own 2-line row. - "Pages under
docs/dev/are … not overflow homes for comment text" gains the distinction it was missing. A cross-package census is not overflow, a long why is;evals/fixtures/testdata/cross-package-census/docs/dev/remote-read-budgets.mdis a fixture standing in for a page of the first kind.
This refines a rule that is days old, not a correction of settled doc. The compression standard the skill already states is the gate author's own, and the count bound is the half that was left unwritten.
The evidence
Measured on !1875 (merged) while taking its banner count back down. That branch went from
62 banners to 166 across its touched Go files with every gate green, because a
merge brought the gate in mid-branch and paragraph-to-banner conversion was the
only compliant edit the text described. It is now at 57, below main's 62 over
the same files. Restoring one file from its merge base removed 19 banners at
zero cost — that is outcome 4, and it was not discoverable from either artifact.
The eval
.claude/skills/write-readable-go/evals/ holds five fixtures. Four put an agent
in the ratchet: a Go file with a legacy over-cap block, and a task requiring a
one-line correction inside it. grade.py scores the outcome chosen, not the
prose — a run of banners and a truncation that drops an external constraint are
both failures.
file-header-banner-run is the exception, on purpose. The gate is silent on a
run of banners, so that fixture forces the edit with a factual error alone and
checks whether the agent collapses the run with no report pushing it. Its
earlier form quoted a gate violation the gate does not produce; review caught
that, and the rewritten fixture is materially harder.
| Eval | What it sets up | Failure it discriminates |
|---|---|---|
unexported-stale-citation |
16-line doc on an unexported function, cap 1, stale symbol citation | compressing away the work-item URL and the ADR reference |
test-claim-owned-by-assertion |
8-line _test.go block, cap 2, claims verbatim in t.Fatalf |
rewording claims the messages already own |
cross-package-census |
11-line comment counting which formats build a remote.SingleFlight |
keeping the census in Go, or compressing the operator-facing facts away |
file-header-banner-run |
seven-banner file header, one banner wrong, no gate report | correcting the wrong banner and leaving the run |
cited-paragraph-leave-and-disclose |
four over-cap docs, three citing the fourth by name | compressing the cited paragraph alone, leaving three dangling pointers |
Pass rate
Sonnet subagents. A pass means every check for that fixture held. main is the
skill as it stands; v1 through v5 are successive drafts, v5 being this MR's head.
The two tables are split because file-header-banner-run is the one fixture the
gate does not drive and the one this round's last edit targeted, so it was run at
six samples rather than three.
The four ratchet fixtures, three runs each
| Fixture | main | v1 | v2 | v3 | v4 | v5 |
|---|---|---|---|---|---|---|
unexported-stale-citation |
0/3 | 0/3 | 1/3 | 3/3 | 3/3 | 3/3 |
test-claim-owned-by-assertion |
3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 |
cross-package-census |
3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 2/3 |
cited-paragraph-leave-and-disclose |
0/3 | 0/3 | 3/3 | 3/3 | 3/3 | 3/3 |
| Subtotal | 6/12 | 6/12 | 10/12 | 12/12 | 12/12 | 11/12 |
file-header-banner-run, the fixture with no gate report
Six runs for v4 and v5, three for the rest, because this is the row the v5 edit was aimed at and three samples cannot separate 1/3 from 2/3.
| main | v1 | v2 | v3 | v4 | v5 |
|---|---|---|---|---|---|
| 0/3 | 2/3 | 1/3 | 2/3 | 4/6 | 6/6 |
The revisions were forced by measured failures, not by taste:
- v1 → v2. v1's fifth outcome read "leave it over the cap and say so in the
MR description", and agents took that as licence to correct a block and leave
it at 16 lines — a commit the hook refuses. v1 buys nothing over
mainoncited-paragraph-leave-and-disclose, 0/3 either way, and it fails differently:main's runs compress the cited paragraph alone and leave three dangling pointers, v1's touch all four blocks and leave them over the cap. v2 says the outcome is declining the edit, and adds that a block other doc comments cite by name comes to the cap with them or is declined with them. - v2 → v3. Two of three runs kept the tracking pointer as a bare
#918, which resolves to nothing from a Go file. v3 says it keeps its full URL. - v3 → v4. Review round. The corrections listed under What changed above.
- v4 → v5.
file-header-banner-runwas failing for one reason across every draft: agents read the banner bound as part of the ratchet procedure and concluded it was dormant with the gate silent. One wrote that the guidance "targets banners used to convert an over-cap legacy block to dodge the comment-cap gate; nothing here is over cap". They were reading the rule correctly as placed — this MR had put the bound inside the ratchet section. v5 lifts it out and says where a run's claims go, which is the other objection the runs raised: "collapsing them now risks discarding six independent, unverified claims to fix one wrong one".
What the banner row does and does not support. All six v5 runs restored the header from the merge base, and several cite the new section while doing it — one wrote "per the write-readable-go skill's 'A run of banners' guidance, a run that is the entire diff gets restored from the merge base". That the text is being read and applied is the strong evidence here. 6/6 against 4/6 is not, on its own: at six samples the two are not far enough apart to call, and the v4 cell also shows why the earlier three-sample reading of this row was noise. The claim this row supports is that the bound now gets applied when the gate is silent, which no earlier draft managed; it is not a claim about the size of the improvement.
The one regression in the other table is cross-package-census at 2/3, one run
dropping the ignored Retry-After from both the Go file and the doc page. The
v5 edit does not touch that outcome, so this reads as noise on a fixture that
sat at 3/3 in five straight columns, but it is a real failing run and is
reported as one rather than smoothed over.
test-claim-owned-by-assertion and cross-package-census pass at 3/3 in every
variant, so they discriminate nothing between drafts. They are kept as
regression cover for the free-slot and docs/dev/ outcomes, not offered as a
result.
Fixture Go files sit under evals/fixtures/testdata/ so the caps gate's own
*/testdata/* filter skips their deliberately over-cap blocks. .gitignore
gains two re-include lines for README.md and grade.py, matching the
evals.json and fixtures/** lines already there.
Grading fidelity
A pass rate is worth only as much as the harness that measured it. Review found five defects in this one, all of the same class — a check that looks right and silently scores the wrong thing — and all five are fixed here.
--selftestnever ran the gate. It compared the helper against three integers frozen in the file, so the README's claim to check "against the verdictscheck-comment-caps.shgives" described something that did not exist, and the constants would rot silently on the next gate change. It now builds a throwaway repository, runs the real gate, and parses the counts.- The helper exempted every banner-opened segment, where the gate exempts only gap-separated ones. Measured: a banner plus one prose line directly above an unexported func is charged 2 by the gate and was counted 0. That scored a banner conversion — the one outcome the skill bans — as a pass.
- A grading run always exited 0, so
outcome_pass: falsewas indistinguishable from a pass to anything scripting the harness. fixture/was both the grader's baseline and the agent's working tree. An agent that edited in place made baseline and answer the same file, sograde_3's "the doc page was edited" check compared an edit against itself. It had already fired: 2 of the 45 stored runs are unrecoverable this way and were re-run.grade.pynow exits 2 without grading whenfixture/is not the fixture it shipped as."revalidate arm" not in gocould not tell a claim from its denial. Every correct fix keeps the phrase and negates it ("and no revalidate arm here"), so the check failed the answers it was meant to accept. It now tests whether the phrase is asserted. Measured over the 27file-header-banner-runruns: the blind form failed 11 of them on that check and changed 1 verdict outright.
The probe set is four files, each sitting deliberately over its cap, because the gate is silent on a block within its cap and would name no count to compare — so the check is agreement above the cap, not at it. Each probe was mutation-checked against the bug it exists for, and each catches exactly one:
ok blank-separated stack joins: counted 3, gate charges 3
ok gap-separated banner keeps its own row: counted 2, gate charges 2
ok an adjacent banner is charged in full: counted 2, gate charges 2
ok a comment naming the symbol is not its declaration: counted 2, gate charges 2lint:skill-scripts:write-readable-go runs --selftest in CI, on the
lint:skill-scripts:bug-report pattern, and lists
scripts/ci/check-comment-caps.sh in its changes: — a gate change is what
makes the helper wrong, and nothing else in CI reads grade.py. It uses the
non-slim Python tag because the selftest shells out to git.
Size
1046 added lines, of which 138 are reviewable prose: 59 in the skill, 47 in
go-style.md, 30 in .gitlab-ci.yml, 2 in .gitignore. The rest is eval
material — 520 in grade.py, 243 in fixtures, 82 in evals.json, 63 in the
evals README — and splitting it out would land a procedure with no evidence it
works, or fixtures with nothing to grade.
The skill section is 59 lines against the 15-20 the change was scoped to. The
overrun is the five outcomes plus their traps, and the banner rule that had to
leave the ratchet section to work at all; putting the traps only in
go-style.md was tried and lost the two failures v2 and v3 fixed, since an
agent reads the skill first and does not always follow the link.
Verification
./scripts/ci/check-comment-caps.sh --base main— OKgrade.py --selftest— 4/4, against real runs of the gate, each probe mutation-checkedglab ci lint— valid- pre-commit chain green (
markdownlint-cli2,lychee,vale,gitlint)
Related to #915 — that issue owns the gate-side question of whether the cap should ratchet differently for a citation-only fix. This MR does not touch the gate; it documents what an author does under the gate as it stands, whichever way that issue lands.