chore(skills): teach write-readable-go the comment-cap ratchet

What changed

.claude/skills/write-readable-go/SKILL.md and docs/dev/go-style.md gain the decision procedure for an edit that lands inside a comment block already over its cap, plus a count bound on the banner carve-out.

The caps gate charges every line of a block once a diff touches any line in it, a pure deletion included, so correcting one sentence inside a 40-line legacy block reports the whole block. Neither artifact said what to do at that point. Both also described the banner carve-out in the singular — "a section banner costs the declaration under it nothing" — without bounding how many you may write, so converting a paragraph into a run of banners was the cheapest compliant edit available.

The five outcomes are all established practice: compress to the cap, relocate to a site with budget, move a cross-package census to docs/dev/ behind a pointer, restore the file from the merge base, or decline the edit and disclose it. The skill carries them as an ordered list; go-style.md carries the same five as a table with the trap on each, so the two do not drift.

The banner bound is deliberately not one of those five, and both artifacts now say so. Its trigger is finding a run, not the gate reporting one, because the gate never will: every gap-separated segment keeps its own row, so a run of ten passes in silence. Both artifacts also now say where a run's claims go — each onto the declaration it describes, which for a file header is usually a declaration in another file, or into docs/dev/ behind a pointer when a claim spans the package. That last part is what makes the bound actionable rather than advisory; see Pass rate for what it did and did not move.

Four smaller gaps close alongside:

  • A test's own strings — an assertion message, t.Skip or t.Fatalf text, a per-case why column — are a free slot neither artifact mentioned, and they surface on failure where a comment does not.
  • A mid-body block was listed among the free slots and is not one: it takes the 2-line cap of any other block, so a 3-line why relocated there is refused a second time. The body head is not a site at all, since it shares the doc comment's cap.
  • A file header needs no banner: a detached block before the import list is attached to no declaration and already takes its own 2-line row.
  • "Pages under docs/dev/ are … not overflow homes for comment text" gains the distinction it was missing. A cross-package census is not overflow, a long why is; evals/fixtures/testdata/cross-package-census/docs/dev/remote-read-budgets.md is a fixture standing in for a page of the first kind.

This refines a rule that is days old, not a correction of settled doc. The compression standard the skill already states is the gate author's own, and the count bound is the half that was left unwritten.

The evidence

Measured on !1875 (merged) while taking its banner count back down. That branch went from 62 banners to 166 across its touched Go files with every gate green, because a merge brought the gate in mid-branch and paragraph-to-banner conversion was the only compliant edit the text described. It is now at 57, below main's 62 over the same files. Restoring one file from its merge base removed 19 banners at zero cost — that is outcome 4, and it was not discoverable from either artifact.

The eval

.claude/skills/write-readable-go/evals/ holds five fixtures. Four put an agent in the ratchet: a Go file with a legacy over-cap block, and a task requiring a one-line correction inside it. grade.py scores the outcome chosen, not the prose — a run of banners and a truncation that drops an external constraint are both failures.

file-header-banner-run is the exception, on purpose. The gate is silent on a run of banners, so that fixture forces the edit with a factual error alone and checks whether the agent collapses the run with no report pushing it. Its earlier form quoted a gate violation the gate does not produce; review caught that, and the rewritten fixture is materially harder.

Eval What it sets up Failure it discriminates
unexported-stale-citation 16-line doc on an unexported function, cap 1, stale symbol citation compressing away the work-item URL and the ADR reference
test-claim-owned-by-assertion 8-line _test.go block, cap 2, claims verbatim in t.Fatalf rewording claims the messages already own
cross-package-census 11-line comment counting which formats build a remote.SingleFlight keeping the census in Go, or compressing the operator-facing facts away
file-header-banner-run seven-banner file header, one banner wrong, no gate report correcting the wrong banner and leaving the run
cited-paragraph-leave-and-disclose four over-cap docs, three citing the fourth by name compressing the cited paragraph alone, leaving three dangling pointers

Pass rate

Sonnet subagents. A pass means every check for that fixture held. main is the skill as it stands; v1 through v5 are successive drafts, v5 being this MR's head. The two tables are split because file-header-banner-run is the one fixture the gate does not drive and the one this round's last edit targeted, so it was run at six samples rather than three.

The four ratchet fixtures, three runs each

Fixture main v1 v2 v3 v4 v5
unexported-stale-citation 0/3 0/3 1/3 3/3 3/3 3/3
test-claim-owned-by-assertion 3/3 3/3 3/3 3/3 3/3 3/3
cross-package-census 3/3 3/3 3/3 3/3 3/3 2/3
cited-paragraph-leave-and-disclose 0/3 0/3 3/3 3/3 3/3 3/3
Subtotal 6/12 6/12 10/12 12/12 12/12 11/12

file-header-banner-run, the fixture with no gate report

Six runs for v4 and v5, three for the rest, because this is the row the v5 edit was aimed at and three samples cannot separate 1/3 from 2/3.

main v1 v2 v3 v4 v5
0/3 2/3 1/3 2/3 4/6 6/6

The revisions were forced by measured failures, not by taste:

  • v1 → v2. v1's fifth outcome read "leave it over the cap and say so in the MR description", and agents took that as licence to correct a block and leave it at 16 lines — a commit the hook refuses. v1 buys nothing over main on cited-paragraph-leave-and-disclose, 0/3 either way, and it fails differently: main's runs compress the cited paragraph alone and leave three dangling pointers, v1's touch all four blocks and leave them over the cap. v2 says the outcome is declining the edit, and adds that a block other doc comments cite by name comes to the cap with them or is declined with them.
  • v2 → v3. Two of three runs kept the tracking pointer as a bare #918, which resolves to nothing from a Go file. v3 says it keeps its full URL.
  • v3 → v4. Review round. The corrections listed under What changed above.
  • v4 → v5. file-header-banner-run was failing for one reason across every draft: agents read the banner bound as part of the ratchet procedure and concluded it was dormant with the gate silent. One wrote that the guidance "targets banners used to convert an over-cap legacy block to dodge the comment-cap gate; nothing here is over cap". They were reading the rule correctly as placed — this MR had put the bound inside the ratchet section. v5 lifts it out and says where a run's claims go, which is the other objection the runs raised: "collapsing them now risks discarding six independent, unverified claims to fix one wrong one".

What the banner row does and does not support. All six v5 runs restored the header from the merge base, and several cite the new section while doing it — one wrote "per the write-readable-go skill's 'A run of banners' guidance, a run that is the entire diff gets restored from the merge base". That the text is being read and applied is the strong evidence here. 6/6 against 4/6 is not, on its own: at six samples the two are not far enough apart to call, and the v4 cell also shows why the earlier three-sample reading of this row was noise. The claim this row supports is that the bound now gets applied when the gate is silent, which no earlier draft managed; it is not a claim about the size of the improvement.

The one regression in the other table is cross-package-census at 2/3, one run dropping the ignored Retry-After from both the Go file and the doc page. The v5 edit does not touch that outcome, so this reads as noise on a fixture that sat at 3/3 in five straight columns, but it is a real failing run and is reported as one rather than smoothed over.

test-claim-owned-by-assertion and cross-package-census pass at 3/3 in every variant, so they discriminate nothing between drafts. They are kept as regression cover for the free-slot and docs/dev/ outcomes, not offered as a result.

Fixture Go files sit under evals/fixtures/testdata/ so the caps gate's own */testdata/* filter skips their deliberately over-cap blocks. .gitignore gains two re-include lines for README.md and grade.py, matching the evals.json and fixtures/** lines already there.

Grading fidelity

A pass rate is worth only as much as the harness that measured it. Review found five defects in this one, all of the same class — a check that looks right and silently scores the wrong thing — and all five are fixed here.

  1. --selftest never ran the gate. It compared the helper against three integers frozen in the file, so the README's claim to check "against the verdicts check-comment-caps.sh gives" described something that did not exist, and the constants would rot silently on the next gate change. It now builds a throwaway repository, runs the real gate, and parses the counts.
  2. The helper exempted every banner-opened segment, where the gate exempts only gap-separated ones. Measured: a banner plus one prose line directly above an unexported func is charged 2 by the gate and was counted 0. That scored a banner conversion — the one outcome the skill bans — as a pass.
  3. A grading run always exited 0, so outcome_pass: false was indistinguishable from a pass to anything scripting the harness.
  4. fixture/ was both the grader's baseline and the agent's working tree. An agent that edited in place made baseline and answer the same file, so grade_3's "the doc page was edited" check compared an edit against itself. It had already fired: 2 of the 45 stored runs are unrecoverable this way and were re-run. grade.py now exits 2 without grading when fixture/ is not the fixture it shipped as.
  5. "revalidate arm" not in go could not tell a claim from its denial. Every correct fix keeps the phrase and negates it ("and no revalidate arm here"), so the check failed the answers it was meant to accept. It now tests whether the phrase is asserted. Measured over the 27 file-header-banner-run runs: the blind form failed 11 of them on that check and changed 1 verdict outright.

The probe set is four files, each sitting deliberately over its cap, because the gate is silent on a block within its cap and would name no count to compare — so the check is agreement above the cap, not at it. Each probe was mutation-checked against the bug it exists for, and each catches exactly one:

ok   blank-separated stack joins: counted 3, gate charges 3
ok   gap-separated banner keeps its own row: counted 2, gate charges 2
ok   an adjacent banner is charged in full: counted 2, gate charges 2
ok   a comment naming the symbol is not its declaration: counted 2, gate charges 2

lint:skill-scripts:write-readable-go runs --selftest in CI, on the lint:skill-scripts:bug-report pattern, and lists scripts/ci/check-comment-caps.sh in its changes: — a gate change is what makes the helper wrong, and nothing else in CI reads grade.py. It uses the non-slim Python tag because the selftest shells out to git.

Size

1046 added lines, of which 138 are reviewable prose: 59 in the skill, 47 in go-style.md, 30 in .gitlab-ci.yml, 2 in .gitignore. The rest is eval material — 520 in grade.py, 243 in fixtures, 82 in evals.json, 63 in the evals README — and splitting it out would land a procedure with no evidence it works, or fixtures with nothing to grade.

The skill section is 59 lines against the 15-20 the change was scoped to. The overrun is the five outcomes plus their traps, and the banner rule that had to leave the ratchet section to work at all; putting the traps only in go-style.md was tried and lost the two failures v2 and v3 fixed, since an agent reads the skill first and does not always follow the link.

Verification

  • ./scripts/ci/check-comment-caps.sh --base main — OK
  • grade.py --selftest — 4/4, against real runs of the gate, each probe mutation-checked
  • glab ci lint — valid
  • pre-commit chain green (markdownlint-cli2, lychee, vale, gitlint)

Related to #915 — that issue owns the gate-side question of whether the cap should ratchet differently for a citation-only fix. This MR does not touch the gate; it documents what an author does under the gate as it stands, whichever way that issue lands.

Edited by Radamanthus Batnag

Merge request reports

Loading
Loading