Add /thorchain/logs endpoint with per-block module log capture

What

Adds an opt-in, node-local API for pulling the log lines modules emitted while executing a range of blocks: the thorchain module by default, cosmos sdk modules on request.

  • Config: thor.api.logs.retain_blocks (env THOR_API_LOGS_RETAIN_BLOCKS). Default 0 disables capture entirely: no files are written and the endpoint returns an error.
  • Capture: log/blocklog.go. Installed at the app's finalize-mode entry points: pre/begin/end blockers and a first ante decorator (baseapp runs messages with the ctx the ante handler returns), so any module logging through ctx.Logger() is recorded. x/bank keeps a logger from construction time, so it gets a window-capture logger keyed on the height opened in PreBlocker and closed in EndBlocker. Every level is recorded regardless of --log_level, so debug lines can be pulled after the fact. Writes are buffered and flushed once per block. One JSONL file per block under <home>/data/thorchain_logs/.
  • Module tagging: entries carry a top-level module lifted from the logger's module field. x/thorchain now tags its BeginBlock, EndBlock and msg handler loggers; untagged finalize-mode lines (thornode's own ante/app code) file under x/thorchain. Side effect: thorchain stdout lines now carry module=x/thorchain.
  • Pruning: EndBlock closes the block's file and removes height - retain_blocks. The first EndBlock after start sweeps stale files left by a restart or retention change and rebuilds the byte total. The directory is capped at 1 GiB, oldest block first, since files share the data disk with the state store.
  • Endpoint: GET /thorchain/logs?start=&end=&level=&modules=, a plain HTTP route on the api server (registered next to /thorchain/supply/cmc), deliberately not a gRPC query: the sdk's local ABCI client shares one mutex across all four cometbft connections, so an ABCI query serializes with FinalizeBlock and a multi-second log scan would stall block execution. These files are node-local, never consensus state, so nothing is gained by routing them through ABCI — and serving directly gives the read a request context to cancel on. start/end default to the last captured block and are clamped to the blocks still on disk, so a range outside that window returns [] rather than an error; malformed or non-positive values are a 400. level defaults to info and includes the more severe levels; modules is repeatable and defaults to x/thorchain. Response is a JSON array of {height, time, level, module, msg, fields}. Bounded by 8 MiB of output, 64 MiB of scanned input and 10000 block files per request, with at most two reads in flight; cap errors name the height they stopped at so the caller can resume. Unauthenticated like the rest of the API; the config comment says so, and also warns about the per-line block-execution cost of enabling capture.

Consensus safety

Nothing here reads or writes KV state. All capture errors are swallowed and logged. With the default config the wrapper is not installed at all, so there is zero behavior change for nodes that do not opt in.

Known limits

  • Only finalize mode is captured; check and simulate are left alone. Exception: x/bank uses the height window, and a gRPC Simulate that races a block can file a bank mint/burn debug line into that block. Narrow and debug-only; noted in code.
  • Ebifrost is not captured: its constructor logger is used from gRPC and subscriber goroutines off the block path.
  • Query-time logs and cometbft logs are not captured, and neither are baseapp's own lines that go through app.logger rather than ctx.Logger() (so an ante-phase panic is missed). No pagination beyond the height named in cap errors.
  • Writes are buffered and flushed on warn/error lines and at block end, so a process crash inside a blocker can lose the last few KiB of that block's debug lines. A block whose file cannot be written is skipped after one error; disabling the feature leaves existing files in place.

Testing

  • go test -tags=mocknet ./log/ ./x/thorchain/... ./app/ ./config/ pass, ./log/ also under -race. Unit tests cover level and module filtering, window capture, exec-mode gating, retention, restart sweep, the byte ceiling, the response and scan caps, the block-count cap, range clamping to the retained window, context cancellation, the read concurrency limit, failed-block skipping, flush on warn/error, capture panic recovery, scanner errors, and replay truncation. x/thorchain/api_logs_test.go drives the HTTP handler through httptest: defaults, level and module selection, clamping, and every rejection path as a 400.
  • Regtest suite test/regression/suites/api/logs.yaml: captured lines carry the right height and module, modules selects and excludes, blocks past retention return empty, a range past the tip is clamped, defaults resolve to the last block at info, and reversed, malformed, zero and unknown-level requests return 400. The scenario emits no x/bank lines, so sdk capture end to end is proven by unit tests, not by regtest.

🤖 Generated with Claude Code

https://claude.ai/code/session_01L9EzUdCvFdQ2pEiWmkcDNw

Edited by Son of Odin

Merge request reports

Loading
Loading