Loading
Add /thorchain/logs endpoint with per-block module log capture
What
Adds an opt-in, node-local API for pulling the log lines modules emitted while executing a range of blocks: the thorchain module by default, cosmos sdk modules on request.
- Config:
thor.api.logs.retain_blocks(envTHOR_API_LOGS_RETAIN_BLOCKS). Default0disables capture entirely: no files are written and the endpoint returns an error. - Capture:
log/blocklog.go. Installed at the app's finalize-mode entry points: pre/begin/end blockers and a first ante decorator (baseapp runs messages with the ctx the ante handler returns), so any module logging throughctx.Logger()is recorded.x/bankkeeps a logger from construction time, so it gets a window-capture logger keyed on the height opened in PreBlocker and closed in EndBlocker. Every level is recorded regardless of--log_level, so debug lines can be pulled after the fact. Writes are buffered and flushed once per block. One JSONL file per block under<home>/data/thorchain_logs/. - Module tagging: entries carry a top-level
modulelifted from the logger'smodulefield.x/thorchainnow tags its BeginBlock, EndBlock and msg handler loggers; untagged finalize-mode lines (thornode's own ante/app code) file underx/thorchain. Side effect: thorchain stdout lines now carrymodule=x/thorchain. - Pruning: EndBlock closes the block's file and removes
height - retain_blocks. The first EndBlock after start sweeps stale files left by a restart or retention change and rebuilds the byte total. The directory is capped at 1 GiB, oldest block first, since files share the data disk with the state store. - Endpoint:
GET /thorchain/logs?start=&end=&level=&modules=, a plain HTTP route on the api server (registered next to/thorchain/supply/cmc), deliberately not a gRPC query: the sdk's local ABCI client shares one mutex across all four cometbft connections, so an ABCI query serializes withFinalizeBlockand a multi-second log scan would stall block execution. These files are node-local, never consensus state, so nothing is gained by routing them through ABCI — and serving directly gives the read a request context to cancel on.start/enddefault to the last captured block and are clamped to the blocks still on disk, so a range outside that window returns[]rather than an error; malformed or non-positive values are a 400.leveldefaults toinfoand includes the more severe levels;modulesis repeatable and defaults tox/thorchain. Response is a JSON array of{height, time, level, module, msg, fields}. Bounded by 8 MiB of output, 64 MiB of scanned input and 10000 block files per request, with at most two reads in flight; cap errors name the height they stopped at so the caller can resume. Unauthenticated like the rest of the API; the config comment says so, and also warns about the per-line block-execution cost of enabling capture.
Consensus safety
Nothing here reads or writes KV state. All capture errors are swallowed and logged. With the default config the wrapper is not installed at all, so there is zero behavior change for nodes that do not opt in.
Known limits
- Only finalize mode is captured; check and simulate are left alone. Exception:
x/bankuses the height window, and a gRPCSimulatethat races a block can file a bank mint/burn debug line into that block. Narrow and debug-only; noted in code. - Ebifrost is not captured: its constructor logger is used from gRPC and subscriber goroutines off the block path.
- Query-time logs and cometbft logs are not captured, and neither are baseapp's own lines that go through
app.loggerrather thanctx.Logger()(so an ante-phase panic is missed). No pagination beyond the height named in cap errors. - Writes are buffered and flushed on warn/error lines and at block end, so a process crash inside a blocker can lose the last few KiB of that block's debug lines. A block whose file cannot be written is skipped after one error; disabling the feature leaves existing files in place.
Testing
go test -tags=mocknet ./log/ ./x/thorchain/... ./app/ ./config/pass,./log/also under-race. Unit tests cover level and module filtering, window capture, exec-mode gating, retention, restart sweep, the byte ceiling, the response and scan caps, the block-count cap, range clamping to the retained window, context cancellation, the read concurrency limit, failed-block skipping, flush on warn/error, capture panic recovery, scanner errors, and replay truncation.x/thorchain/api_logs_test.godrives the HTTP handler throughhttptest: defaults, level and module selection, clamping, and every rejection path as a 400.- Regtest suite
test/regression/suites/api/logs.yaml: captured lines carry the right height and module,modulesselects and excludes, blocks past retention return empty, a range past the tip is clamped, defaults resolve to the last block at info, and reversed, malformed, zero and unknown-level requests return 400. The scenario emits nox/banklines, so sdk capture end to end is proven by unit tests, not by regtest.
Edited by Son of Odin