perf(bifrost): skip never-stored signed-tx deletes, cache the signer keyring record, reuse UTXO coinbase (behavior changes)

Three behavior-changing perf follow-ups. Each needs an explicit OK. The commits are independent, so drop any you don't want. They are the items from the round-2 profile pass that weren't exactly equivalent but carry value beyond CPU. The other three candidates (height-only block-meta prune, Tron height decode, receipt worker pool) were dropped as not worth a review for their small gains.

What changes, exactly

# Change Why it's worth it Benchmark The behavior difference
1 RemoveSignedTxItem checks Has before Delete The EVM/ETH scanners call it for every tx in every block. Each call wrote a leveldb delete marker that GetSignedTxItems then iterated past: ~0.6 s of the 48.6 s production profile. −78 % time, −45 % bytes Removing a hash that is not stored no longer attempts the write. On a database that rejects writes (read-only, or a persistent write error) it returns nil where it returned the error; the scanners only logged that error. Stored hashes and Has errors behave as before.
2 Keys.GetSignerInfo caches the record A file-keyring decrypt (PBKDF2, ~2–3 ms) ran about once a second via sendDeck: 0.26 s plus ~22 M objects (~4.5 % of all allocations). It also removes a crash path: a transient keyring read error used to panic bifrost. 2.3 ms → 0.46 µs, 16,600 → 1 allocs A keyring file changed while bifrost runs isn't re-read (the private key was already read only once, at startup). A keyring error after the first successful read no longer panics; a failing first read still does.
3 UTXO: reuse the scanned block's coinbase value for its confirmation count Removes a second full verbose getblock per observed block on BTC, LTC, BCH, DOGE and ZEC, which halves that load on the chain daemons. The code had a TODO for exactly this. −50 % time, 2 → 1 getblock When that second getblock would have failed transiently, the lookup now returns the already-scanned value instead of an error (after which the caller used the chain's default coinbase). The value is reused only for the same block hash, so a reorged height is fetched again. Errors for uncached blocks are unchanged.

Expected production effect: about 1 s of the profile's 48.6 s (~2 % of bifrost CPU) and ~5 % fewer allocated objects. On UTXO chains, half the heavy getblock RPCs per observed block. These are estimates from the profile, not measured on a validator.

Tests

Each behavior difference is pinned by a test that fails on the old code. Each normal path is checked against the original.

  1. Signed-tx removal:
    • a differential test runs 5,000 random add, remove and list operations against the original Delete-only removal;
    • TestRemoveSignedTxItemReadOnlyDB pins the difference (on the old code it fails with leveldb: read-only mode);
    • a mutant that skips the delete for present keys too is caught.
  2. Keyring: a counting keyring checks:
    • the record is read once, and equals a direct read;
    • a later keyring failure doesn't panic, while a failing first read does;
    • 16 goroutines can read concurrently under -race. The old code fails with "keyring read 2 times".
  3. UTXO coinbase: against a fake node:
    • a scanned block's coinbase lookup makes no getblock call and returns the uncached value;
    • a different hash at the same height is fetched again;
    • a failing getblock returns the cached value (the difference) but still errors, with the same wrapping, for an unscanned block;
    • the cache is bounded to 64 heights. On the old code it fails with getblock calls 2 -> 3.

Suites: go test -tags mocknet on shared/evm, evm, ethereum, thorclient/..., signer/..., observer and utxo/... all pass, as do thorclient and utxo without the tag. The new tests pass under -race. golangci-lint v1.63.4 (repo config) and gofumpt are clean, and go build ./... passes.

Benchmarks

-count 10, -tags mocknet, AMD Ryzen Threadripper 3960X. Each baseline is the benchmark commit.

1. Skip deletes of never-stored signed-tx hashes

                  │     base     │                 new                 │
                  │    sec/op    │   sec/op     vs base                │
SignedTxScan-48     37.774m ± 9%   8.491m ± 9%  -77.52% (p=0.000 n=10)

                  │     base     │                 new                  │
                  │     B/op     │     B/op      vs base                │
SignedTxScan-48     9.700Mi ± 0%   5.371Mi ± 0%  -44.62% (p=0.000 n=10)

                  │    base     │                  new                  │
                  │  allocs/op  │  allocs/op   vs base                  │
SignedTxScan-48     81.83k ± 0%   91.80k ± 0%  +12.18% (p=0.001 n=10)

2. Cache the signer keyring record (file-backed keyring)

                 │      base       │                 new                 │
                 │     sec/op      │   sec/op     vs base                │
GetSignerInfo-48   2307602.0n ± 4%   457.2n ± 1%  -99.98% (p=0.000 n=10)

                 │      base      │                 new                 │
                 │      B/op      │    B/op     vs base                 │
GetSignerInfo-48   549806.50 ± 0%   24.00 ± 0%  -100.00% (p=0.000 n=10)

                 │      base      │                new                 │
                 │   allocs/op    │ allocs/op   vs base                │
GetSignerInfo-48   16600.000 ± 0%   1.000 ± 0%  -99.99% (p=0.000 n=10)

3. UTXO coinbase reuse (112-tx BTC fixture)

                    │    base     │                 new                 │
                    │   sec/op    │   sec/op     vs base                │
ScanThenCoinbase-48   24.57m ± 3%   12.25m ± 9%  -50.14% (p=0.000 n=10)

                    │       base        │                    new                    │
                    │ getblock-calls/op │ getblock-calls/op  vs base                │
ScanThenCoinbase-48          2.000 ± 0%          1.000 ± 0%  -50.00% (p=0.000 n=10)

                    │     base     │                 new                  │
                    │     B/op     │     B/op      vs base                │
ScanThenCoinbase-48   2.724Mi ± 1%   1.376Mi ± 1%  -49.50% (p=0.001 n=10)

                    │    base     │                 new                 │
                    │  allocs/op  │  allocs/op   vs base                │
ScanThenCoinbase-48   7.058k ± 0%   3.669k ± 0%  -48.02% (p=0.001 n=10)

🤖 Generated with Claude Code

Edited by Son of Odin

Merge request reports

Loading
Loading