perf(bifrost): skip never-stored signed-tx deletes, cache the signer keyring record, reuse UTXO coinbase (behavior changes)
Three behavior-changing perf follow-ups. Each needs an explicit OK. The commits are independent, so drop any you don't want. They are the items from the round-2 profile pass that weren't exactly equivalent but carry value beyond CPU. The other three candidates (height-only block-meta prune, Tron height decode, receipt worker pool) were dropped as not worth a review for their small gains.
What changes, exactly
| # | Change | Why it's worth it | Benchmark | The behavior difference |
|---|---|---|---|---|
| 1 | RemoveSignedTxItem checks Has before Delete |
The EVM/ETH scanners call it for every tx in every block. Each call wrote a leveldb delete marker that GetSignedTxItems then iterated past: ~0.6 s of the 48.6 s production profile. |
−78 % time, −45 % bytes | Removing a hash that is not stored no longer attempts the write. On a database that rejects writes (read-only, or a persistent write error) it returns nil where it returned the error; the scanners only logged that error. Stored hashes and Has errors behave as before. |
| 2 | Keys.GetSignerInfo caches the record |
A file-keyring decrypt (PBKDF2, ~2–3 ms) ran about once a second via sendDeck: 0.26 s plus ~22 M objects (~4.5 % of all allocations). It also removes a crash path: a transient keyring read error used to panic bifrost. |
2.3 ms → 0.46 µs, 16,600 → 1 allocs | A keyring file changed while bifrost runs isn't re-read (the private key was already read only once, at startup). A keyring error after the first successful read no longer panics; a failing first read still does. |
| 3 | UTXO: reuse the scanned block's coinbase value for its confirmation count | Removes a second full verbose getblock per observed block on BTC, LTC, BCH, DOGE and ZEC, which halves that load on the chain daemons. The code had a TODO for exactly this. |
−50 % time, 2 → 1 getblock |
When that second getblock would have failed transiently, the lookup now returns the already-scanned value instead of an error (after which the caller used the chain's default coinbase). The value is reused only for the same block hash, so a reorged height is fetched again. Errors for uncached blocks are unchanged. |
Expected production effect: about 1 s of the profile's 48.6 s (~2 % of bifrost CPU) and ~5 % fewer allocated objects. On UTXO chains, half the heavy getblock RPCs per observed block. These are estimates from the profile, not measured on a validator.
Tests
Each behavior difference is pinned by a test that fails on the old code. Each normal path is checked against the original.
- Signed-tx removal:
- a differential test runs 5,000 random add, remove and list operations against the original
Delete-only removal; TestRemoveSignedTxItemReadOnlyDBpins the difference (on the old code it fails withleveldb: read-only mode);- a mutant that skips the delete for present keys too is caught.
- a differential test runs 5,000 random add, remove and list operations against the original
- Keyring: a counting keyring checks:
- the record is read once, and equals a direct read;
- a later keyring failure doesn't panic, while a failing first read does;
- 16 goroutines can read concurrently under
-race. The old code fails with "keyring read 2 times".
- UTXO coinbase: against a fake node:
- a scanned block's coinbase lookup makes no
getblockcall and returns the uncached value; - a different hash at the same height is fetched again;
- a failing
getblockreturns the cached value (the difference) but still errors, with the same wrapping, for an unscanned block; - the cache is bounded to 64 heights.
On the old code it fails with
getblock calls 2 -> 3.
- a scanned block's coinbase lookup makes no
Suites: go test -tags mocknet on shared/evm, evm, ethereum, thorclient/..., signer/..., observer and utxo/... all pass, as do thorclient and utxo without the tag. The new tests pass under -race. golangci-lint v1.63.4 (repo config) and gofumpt are clean, and go build ./... passes.
Benchmarks
-count 10, -tags mocknet, AMD Ryzen Threadripper 3960X. Each baseline is the benchmark commit.
1. Skip deletes of never-stored signed-tx hashes
│ base │ new │
│ sec/op │ sec/op vs base │
SignedTxScan-48 37.774m ± 9% 8.491m ± 9% -77.52% (p=0.000 n=10)
│ base │ new │
│ B/op │ B/op vs base │
SignedTxScan-48 9.700Mi ± 0% 5.371Mi ± 0% -44.62% (p=0.000 n=10)
│ base │ new │
│ allocs/op │ allocs/op vs base │
SignedTxScan-48 81.83k ± 0% 91.80k ± 0% +12.18% (p=0.001 n=10)2. Cache the signer keyring record (file-backed keyring)
│ base │ new │
│ sec/op │ sec/op vs base │
GetSignerInfo-48 2307602.0n ± 4% 457.2n ± 1% -99.98% (p=0.000 n=10)
│ base │ new │
│ B/op │ B/op vs base │
GetSignerInfo-48 549806.50 ± 0% 24.00 ± 0% -100.00% (p=0.000 n=10)
│ base │ new │
│ allocs/op │ allocs/op vs base │
GetSignerInfo-48 16600.000 ± 0% 1.000 ± 0% -99.99% (p=0.000 n=10)3. UTXO coinbase reuse (112-tx BTC fixture)
│ base │ new │
│ sec/op │ sec/op vs base │
ScanThenCoinbase-48 24.57m ± 3% 12.25m ± 9% -50.14% (p=0.000 n=10)
│ base │ new │
│ getblock-calls/op │ getblock-calls/op vs base │
ScanThenCoinbase-48 2.000 ± 0% 1.000 ± 0% -50.00% (p=0.000 n=10)
│ base │ new │
│ B/op │ B/op vs base │
ScanThenCoinbase-48 2.724Mi ± 1% 1.376Mi ± 1% -49.50% (p=0.001 n=10)
│ base │ new │
│ allocs/op │ allocs/op vs base │
ScanThenCoinbase-48 7.058k ± 0% 3.669k ± 0% -48.02% (p=0.001 n=10)