feat(invariants): vault_commitments and asset_backing invariant routes
🔎 Visual review: building it for0e397845; the link appears here.
What and why
Most of the retrospective's state-accounting bugs broke one identity that nothing checked. On one side are the Asgard vault ledgers: the L1 coins THORChain thinks each vault holds. On the other is what is owed against them.
The existing checks don't cover it:
- The invariant routes check native coins against module balances (
asgard,bond, ...). - The bifrost solvency attestation checks the vault ledgers against the L1 chains.
- Nothing checked the vault ledgers against the pools and the outbounds.
This MR adds two query-time invariant routes in x/thorchain/keeper/v1/invariants_backing.go. The regression harness and the simulation watcher already poll every route after each block, so both pick them up automatically.
vault_commitments
Each Asgard vault, of any status, must hold on its ledger at least the coins of the unsigned outbounds assigned to it. The outbound window is the same one GetPendingOutbounds uses, which is what vault selection deducts: each item's own signing period back, TxOutDelayMax forward, plus stale migrate markers.
It fires as soon as a vault is over-committed, before the outbounds sign and debit the ledger past zero. It compares only the outbound coin, not MaxGas, because the last migration round assigns the whole balance on purpose.
Bugs it targets:
- !4803 (merged): refundTx forced a refund onto the inbound vault without deducting that vault's pending outbounds.
- !4802 (merged): the precondition for the silent SubFunds clamp, which left about 270 ETH of phantom credit. Once the clamp has happened, the state shows only a surplus, and no state invariant can see that.
- !4876 (merged): a stale migrate plus a new round sized from the full balance over-committed the retiring vault.
asset_backing
For each L1 pool asset, the active-vault ledgers must cover everything owed in that asset:
- pool
BalanceAsset + PendingInboundAsset; - trade and secured depth;
- queued swaps (the unswapped L1 deposit plus unsettled non-synth output);
- unsigned outbounds of non-inactive vaults.
Only a shortfall breaks it, with zero tolerance. Rules:
- MaxGas is not owed, because gas debits the vault and the pool (through the gas manager) in the same block.
- Coins on inactive vaults don't count as backing.
- The check is skipped while any vault is retiring, because coins migrating between vaults sit on neither ledger.
calculateNetworkSolvencyskips churn the same way. - Recorded deficits appear in the message but are not compared. They already raise a security event and a halt, and the existing
vault_deficitroute covers them. A deficit later settled out of new inbounds still trips the check, because the pool is credited with the full inbound while the vault only gains the remainder. - Pools on a chain that no Asgard vault lists, and whose asset no vault has ever carried, are skipped. That state only exists in hand-written genesis.
Bugs it targets:
- !3408 (merged): an inactive-vault refund fee was credited to the pool while the coins stayed on the inactive vault (the ETH.RAZE insolvency).
- Over-solvency clearing (805020bf; fixes !4454 (merged), !4640 (merged), !4951 (merged)): the solvency calculation missed some liabilities, so the clearing swap credited the pool with coins that already backed something else.
asset_backingimplements the liability model ofcalculateNetworkSolvencyseparately, so a future error in either one shows up as a disagreement.
bond (existing)
The existing bond route already catches !4574 (merged), where the MsgMimir fee floored the bond at zero but sent the full fee out of the bond module. Only a test was missing; this MR adds it.
Harness
create-blocks gains skip_invariant_routes: [route...]. It skips only the named routes, unlike skip_invariants, which skips all of them. Unknown route names fail the op.
solvency/solvency.yaml uses it for one block. Its genesis pool already includes a 200M BTC noop that is only observed in the next block.
Consensus
There are no consensus code changes. Both invariants are read-only (Get* calls and iterators only). They do their own outbound walk rather than calling or changing GetPendingOutbounds.
Note for reviewers: on develop, invariants are not purely query-time. emitSolvencyTelemetry runs every route in EndBlock every 600 blocks on nodes with telemetry enabled. That path is read-only and recovers from panics. The crisis module is not wired (RegisterInvariants is empty). This MR adds both new routes to that sweep's skip list, so they run only at query time.
Testing
- Unit tests in
keeper/v1/invariants_test.gobuild the corrupted state for each case and assert the exact message:asset_backing: !3408 (merged) refund fee; over-solvency over-sweep combining the !4454 (merged)/!4640 (merged)/!4951 (merged) liabilities (streaming output, past-height outbound, trade depth); deficit settled out of a later inbound; unserved pool; balanced and surplus states; retiring skip.vault_commitments: !4803 (merged) forced refund; !4876 (merged) stale migrate; !4802 (merged) clamped vault; signed and expired items ignored.bond: !4574 (merged) mimir fee.- Command:
go test -tags=mocknet ./x/thorchain/ ./x/thorchain/keeper/..., all pass.
- All 194 regression suites:
make test-regression CI=1 EXPORT=1 DEBUG=1, run in a uniquely tagged container. Results: 194 passed, 0 retried. No change undertest/regression/mnt/. make lint-ci: clean.- Not run: the simulation (
make test-simulation). Its invariant watcher polls the new routes too, so a first sim run is the real-daemon check. See Follow-ups.
What the first full regression run found, before the definition was settled. asset_backing tripped in 7 suites; vault_commitments never tripped. None was a code bug:
- 4 eddsa suites (
xmr,xmr-add-liquidity,xmr-errata-before-quorum,xmr-spent-refs): genesis has BTC and ETH pools, but the only vault is an XMR monovault that serves neither chain. Fixed by the unserved-pool skip. gas/outbound-over-max-gas-avax(line 179) andvault-deficit/deficit-tracking(line 49): the suites deliberately drive a vault past zero, and the deficit is booked and halted on. Fixed by not comparing deficits.solvency/solvency(block 1): deliberate genesis gap that the next block closes. Handled withskip_invariant_routes.
Mainnet today
asset_backing would trip on mainnet now, by dust amounts. This is an estimate: the live /thorchain/vaults/solvency figure (which reserves MaxGas) plus the MaxGas of the queued outbounds, measured on 2026-11-16 at height ~28.70M. There are no retiring vaults and no deficits on mainnet.
| Asset | Shortfall | Pool depth | Share of pool |
|---|---|---|---|
| BASE.ETH | 664,042 (0.0066 ETH) | 10.89 ETH | 0.06% |
| BSC.BNB | 20,000 (0.0002 BNB) | 2,205 BNB | negligible |
| ETH.WSTETH | 141 (1.4e-6) | none listed | — |
Nothing polls the routes on mainnet, and they are kept out of telemetry, so this has no operational effect. It does mean a mainnet-fork replay would need a tolerance of about 0.07% of pool depth, or a per-chain dust allowance. All other assets have a surplus.
Not covered
- !4730 (merged) SlashVault under-slashing multi-coin thefts: the slash amount is wrong, but every ledger still balances. This needs a transition check (slash value ≥ stolen value), not a state invariant. Its "pool credited when the slash transfer failed" half is already caught by
asgard. - The phantom credit left after the !4802 (merged) clamp, which shows as surplus. That is the L1 attestation's job.
- Backing during churn: skipped while any vault is retiring.
- Recorded deficits, until they are settled.
- Gas-only over-commitment: MaxGas is excluded from
vault_commitments. - Vault ledger vs L1 drift (!4371 (merged), !4697 (merged), !5244 (merged)): the books balance in those cases, so they need the bifrost solvency attestation.
Follow-ups
- Run the simulation with these routes. The bifrost observation paths for failed outbounds, gas and XMR may expose cases regression does not reach.
- Measure the cost of the outbound-window walk on mainnet-sized state, then add both routes to the telemetry sweep and alert on
thornode_invariant_broken{invariant="asset_backing"}. - Track in-flight migrations, so
asset_backingcan run during churn. - Add an x/thorchain test that checks
calculateNetworkSolvencyagainstasset_backingon random states.