feat(skills): add tc-metrics for Midgard metric anomaly triage

🔎 Visual review of 8f53933f: explainer, review flags and prompts to ask an AI about the change

Adds a tc-metrics Claude Code skill on top of the anomaly-scoring endpoints from midgard!733 (merged).

.claude/skills/tc-metrics/metrics.py ranks every series (736 today) by how unusual its newest hour is. Stale rows come first, then commonness ascending, then |modifiedZ| descending. --grep restricts the ranking to matching series, and --series prints one series' full hourly history. The script also handles the things that make the raw endpoint misleading in practice:

Trap Handling
/anomalies returns only samples=0 series while history is short: most series' 2/n floor sits above the 0.05 default until n ≥ 40 (and 24h is rejected outright below 0.083) Detected from the actual bucket counts and reported in the header; ranks /scores directly instead
bucketCount is per-series, so commonness is not comparable across rows Header prints the bucket range and the most common n; exposed as an n column; n < 8 flagged
MAD collapses to ~0 on near-constant series, giving |z| up to 1e14 Flagged, never dismissed: "was flat, moved this hour" when deltaCommonness is also low, otherwise "level shifted within window; --series for when"
samples = 0 means that series' source failed, and ties push commonness up, so a frozen series scores as increasingly healthy Flagged STALE, counted in the header and sorted above the ranking so --top cannot hide it
thornode_invariant_broken / thornode_solvency_excess_e8 mean something regardless of rank Shown unconditionally in a "read by value, not by rank" block
A 404 means collection is disabled on that Midgard, not that the network is fine Reported as such; no fallback host

SKILL.md splits series into three groups:

  • Node-local: go_*, process_*, promhttp_*, and the p2p/mempool/blocksync/consensus_step_duration series describe only the Midgard host's thornode.
  • Seen by one node: the other tendermint_/cometbft_ consensus series need corroboration before they count as network findings.
  • Network-wide: thornode_* (all four families come from THORNode REST, not the Prometheus scrape, so they go stale independently of it), swap_*, pool_* and liquidity_* (Midgard DB).

It also covers:

  • the ~2h staleness, so findings are framed as "what was unusual last hour" and not "what is happening now";
  • the hour-average value trap: confirm levels against the THORNode API before quoting them;
  • the row shapes that need a second look: per-proposer delays, mimirs, monotone trends like tendermint_consensus_height.

health gets an optional step 1b discovery sweep. It runs metrics.py with a grep that drops node-local and per-proposer series, explains how to read the output, uses survivors to narrow the Loki windows, and adds a "Metric Anomalies" report table. Any network-wide survivor that stays unexplained makes the report at least 🟡. A 404 from a Midgard without !733 (merged) is noted as "unavailable" and the step is skipped. tc-debug gets a one-line pointer here for the "no specific target, something just feels off" case.

Also bumps the Go toolchain from 1.26.9 to 1.26.10 for GO-2026-6809: crypto/x509 now distrusts compromised intermediate certificates, and govulncheck reports this as reachable through the monero sidecar's net/http server. The bump covers go.mod, .go-version, CI and Docker images, trunk, scripts and docs, plus CI_TOOLS_VERSION so the tools image gets rebuilt.

Verified end to end against https://gateway-lon.liquify.com/chain/thorchain_midgard/v2/metrics.

🤖 Generated with Claude Code

https://claude.ai/code/session_01CM7iDAf5ZWaJf7u8seWtYv

Edited by Huginn Ai

Merge request reports

Loading
Loading