Draft: perf(thorchain): EndBlock also loads the scheduled txout heights with one iteration

Stacked on !5280 (merged) (the past txout window). This MR targets odin/endblock-txout-window, so its diff shows only the two commits on top. GitLab retargets it to develop when !5280 (merged) merges.

Summary

!5280 (merged) loads the past txout heights (h-900 .. h-1) with one store iteration in EndBlock. Two EndBlock paths also scan the scheduled heights, from the current one up to MaxTxOutOffset (720) ahead, plus the first height past that:

  • GetPendingOutbounds, for each L1 outbound.
  • migrateFunds, every block while there are retiring vaults (during churn).

Nearly all of those heights are empty. As with the past heights, the first Has of each absent height in a block falls through to a goleveldb miss, because the inter-block cache only keeps values that exist.

This attaches a second TxOutWindow in AppModule.EndBlock for h+1 .. h+MaxTxOutOffset+1, over the past one. Windows attached over each other stay independent:

  • A read uses the innermost window covering its height.
  • A write disables only the windows covering it. A delayed outbound written at a scheduled height disables the scheduled window for the rest of the block and leaves the past one.

The current height h is in neither window, because EndBlock keeps appending outbounds to it.

Risk: high (keeper and EndBlock). State, events and gas_used are unchanged; see Evidence.

What changes

  • keeper/txout_window.go:
    • A window keeps a pointer to the window that was on the ctx when it was attached (outer).
    • New Covering(height) returns the innermost enabled window covering a height.
    • NoteWrite now goes through the whole chain.
  • keeper/v1/keeper_txout.go: getTxOutWindowBytes reads through Covering.
  • module.go: attaches the scheduled window after the past one.

Consensus and gas

As in !5280 (merged), the windows are attached only in AppModule.EndBlock, whose gas is not part of any tx result. Txs and queries never see them.

I had a reviewer subagent try to break it. It checked every txout writer, contexts that drop the windows, discarded and committed cache contexts, digit-count edges, MaxTxOutOffset changing mid-block, and overflow. It found nothing. It also ran a randomized test (300 seeds of random reads, sets, appends and clears near digit-count boundaries, through nested cache contexts that are randomly committed or discarded): every windowed read equalled a raw store read. That test caught a deliberately broken invalidation in each window within two seeds.

Evidence

Mainnet replay (scripts/perf/mainnet-replay.sh, 1,999 blocks 28124001..28125999 through the CometBFT handshake, which checks every block's app hash against mainnet). The range includes a churn (retiring vaults), so migrateFunds scans every block. This change was combined with !5283 (merged) and !5285 (merged) on the v3.20.3-based replay lab stack (which includes !5280 (merged)), 3 interleaved rounds per side. This change's part, from CPU profiles summed over the 3 rounds:

before after Δ
getTxOut (store path) 13.4 s 5.8 s −57%
getTxOutWindowBytes 4.4 s 5.7 s +1.3 s
GetTxOut 22.3 s 15.5 s −31%
migrateFunds 13.6 s 7.1 s −48%
GetPendingOutbounds 3.6 s 2.2 s −39%

That is about 1.3% of FinalizeBlock CPU (536.8 s). Outside churn migrateFunds returns early, so the gain is the GetPendingOutbounds part plus whatever else reads the scheduled heights in a block. In blocks with a delayed outbound, the window only serves reads made before that write.

scripts/perf/guard.sh (unit tests, then unfiltered regtest vs !5280 (merged)'s head): PASS. All 2,481 regression output files are identical (state, events, gas_used).

Tests

keeper/v1/keeper_txout_window_test.go TestStackedWindows, with a past window 100..199 and a scheduled window 201..300:

  • Reads at heights 90..310 equal plain reads.
  • A raw store write the windows can't see is still served from each window's snapshot.
  • Appending to the current height (200) keeps both windows.
  • A scheduled outbound at 260, made in a discarded cache context, disables only the scheduled window. The past window still serves 150.
  • A write at 120 then disables the past window.

The test fails if reads don't walk the chain.

🤖 Generated with Claude Code

Merge request reports

Loading
Loading