perf(app): node-local per-height response cache for external queries

🔎 Visual review of f08ee5b0: explainer, review flags and prompts to ask an AI about the change

Summary

Frontends and gateways poll pools, inbound_addresses, network, nodes, vaults, wasm smart queries and quotes many times per ~6 s block. A mainnet query node spent ~26% of its CPU recomputing identical responses (production profile, 2026-10-07).

This adds a node-local, in-memory response cache for external queries. It's keyed by (entrypoint, height, method, request bytes), with singleflight coalescing and a byte-bounded LRU.

Risk: medium (API responses, plus an end-of-Commit hook in app/app.go). It changes no state or consensus behavior; see below.

How it works

  • Two entrypoints only:
    • THORChainApp.Query overrides BaseApp.Query. That covers CometBFT RPC abci_query, and REST via ABCI.
    • The gRPC server registration (abciParityServer) covers external gRPC, and REST via the gateway, which calls the node's own gRPC server.
  • Heights: a request without a height resolves to the last height marked ready by the end-of-Commit hook (PrepareCheckStater), and that height goes into the key. A new block therefore always selects new keys. "Latest" entries of older heights are dropped on commit. Explicit heights above ready go through uncached.
  • Stored responses: only successful responses are stored. Errors are neither cached nor shared with singleflight waiters.
  • gRPC entries are stored as wire bytes, and each caller decodes its own message (213e49475f). Nothing a caller changes reaches another caller, and the byte cap charges what an entry holds. No client sees a difference: gRPC clients, and REST through the gateway's client connection, already receive the response decoded from the wire.
  • Cached services: types.Query, cosmwasm.wasm.v1.Query, bank, auth, denom, scheduler. Excluded: Block, Export, and keysign/keygen (signed with the node keyring).
  • Quotes are served from cache only at the latest height, and only while that block is fresh by the same thor.api.quote.max_lag rule the quote guard applies.

Consensus

Queries from inside the state machine never pass through either entrypoint. CosmWasm's gRPC/Stargate query plugins resolve handlers from BaseApp.GRPCQueryRouter() directly, and modules call keepers. So in-tx store reads and gas_used are untouched. TestQueryCacheBypassedInsideStateMachine plants a poisoned entry that the ABCI path would serve, and checks that a wasm query still reads the store, with gas equal to the no-cache run.

Config (off-chain, default on)

thor.api.query_cache.enabled=true and max_bytes=128MiB (env THOR_API_QUERY_CACHE_*), documented in config/default.yaml. Wired in newApp, so appExport never gets a cache.

Evidence

  • scripts/perf/guard.sh vs develop: PASS. All 2,481 regression output files are identical, including state, events and gas_used.
  • Benchmarks (replay fixture, ABCI): pools miss 517 µs / hit 0.40 µs; network 249 / 0.39 µs; nodes 190 / 0.37 µs.
  • Not yet measured: the hit rate and CPU saving under real mainnet query load. That needs a profile on a query node after deploy.

Tests and coverage

  • app/query_cache_test.go covers:
    • policy, readiness and the byte cap
    • the commit sweep
    • singleflight
    • errors not cached or shared
    • ABCI hits identical to uncached
    • new height never stale
    • exclusions
    • the state-machine bypass
    • gRPC hits, headers and bursts
    • gateway JSON of a miss and a hit equal to the uncached response after a wire round trip
    • a caller's changes not reaching the next caller
  • All pass with -race.
  • Changed-line coverage: app 217/241. The 5 uncovered cmd/thornode lines are startup wiring in newApp. In total that's 217/246 (88%).

Reviewer notes

  • Queriers that read in-memory manager state rather than the store (for example mgr.GetVersion() in quote asset parsing) are pinned per height. That only matters across a version change, and uncached responses at explicit old heights already had that skew.
  • Cache hits skip BaseApp's query telemetry counters. Hits and misses are counted as query_cache.hit and query_cache.miss instead.

🤖 Generated with Claude Code

Edited by Huginn Ai

Merge request reports

Loading
Loading