perf(bifrost/eth): skip eth_getBlockReceipts when logsBloom rules out observations
🔎 Visual review of6a4b5b43: explainer, review flags and prompts to ask an AI about the change
🔎 Visual review: building it for6a4b5b43; the link appears here.
Problem
The ETH scanner calls eth_getBlockReceipts for every block with txs, but most blocks have nothing THORChain needs from their receipts. A receipt only leads to an observation through a log emitted by a router, or a tx to or from a vault. In the two 600-block mainnet windows below, 57 and 212 blocks had a router log, and the receipts averaged about 870 KB and 960 KB per block.
!5321 (merged) added a filter for this to the BSC scanner. ETH has its own client, so it needs its own.
Changes
receiptsMayProduceObservation(new):extractTxscalls this first and skips the receipts call only if all of these hold:- no active router from
GetContractsis in the block'slogsBloom - no tx is to a vault or from a vault
- no tx is a
transfer(address,uint256)to a vault
- no active router from
- Anything it can't settle fetches as before: no routers known, an empty bloom on a block with txs, a sender that is neither reported nor recoverable, or truncated
transfercalldata. - Senders as in !5321 (merged): the filter uses the sender the RPC reported, which
ethclientkeeps with the tx, and falls back to the signature without making a call. - Skipped blocks behave the same otherwise: they clear the same signed tx items, passing over blob txs and contract creations as
extractTxsdoes. Gas prices don't change, becauseprocessBlockreads them from tx fields before this runs.
How this compares with the BSC filter
The rules match !5321 (merged) plus the changes proposed in !5330 (merged). That MR changes a different package, so neither depends on the other.
- Txs to known contracts don't fetch: the log parser only reads logs from contracts the filter has already tested in the bloom, so a tx to a router or whitelisted aggregator can't be observed once they're ruled out. Without this, about a third of the blocks the filter skips would fetch: in a 300-block sample, 57 of the 161 blocks with no bloom hit fetched only because of a whitelisted aggregator.
- An empty bloom fetches: an all-zero bloom on a block with txs may mean nothing filled it in, and fetching is correct either way.
Two things are specific to ETH:
- Whitelisted emitters: with
useWhitelistSmartContracton, ETH's log parser also accepts logs from whitelisted contracts, so the filter tests those in the bloom too. - Gas prices: this filter only decides whether to fetch.
processBlockhas already read gas prices from tx fields, so it doesn't work them out as the BSC filter does.
Live comparison
I ran the filter and the receipts path side by side on two windows of 600 consecutive mainnet blocks, with the 6 active and 41 inactive vaults, the default token list and whitelist, and both EVMDisableContractWhitelist and direct ERC20 inbounds on. For every skipped block, the receipts path had to produce no observations.
| Window | Skipped | Fetched | Observations | Blocks with a router log | Bloom false positives | Missed observations |
|---|---|---|---|---|---|---|
| 26344678..26345277 | 318 (53%) | 282 | 68 | 57 | 222 | 0 |
| 26345670..26346269 | 234 (39%) | 366 | 296 | 212 | 151 | 0 |
- Observations: the receipts path produced 364, all in fetched blocks: 179 from vaults through the router, 153 deposits to the router, 15 sends to vaults and 17 through other contracts.
- Router logs: all 269 blocks with a router log hit the bloom.
- False positives: about 40% of the blocks with no router log hit the bloom anyway. ETH blooms are dense, about 1,400 of 2,048 bits set in a 300-block sample, so this is the main limit on the filter.
- Savings: 39-53% of the receipts calls and 32-46% of the receipt data, because busier blocks fetch more often. The skip rate falls as router activity rises.
- Filter cost: median 0.01 ms per block and p99 0.77 ms in the second window; most blocks that fetch return on the bloom before reading a sender.
- Reported senders: on 50 further blocks, all 13,975 senders were available without a call and all matched the signature. Reading them took 0.2 ms per block at the median, against 17 ms for recovering them.
These windows cover about 4 hours of blocks in total.
Tests
receipts_gate_test.go adds 16 tests. One replays 300 random blocks with truthful blooms, once with the filter and once forced down the receipts path, and requires the same observations with the contract whitelist on and off. I broke the implementation 25 different ways, one at a time, and the tests caught each one.
Not covered
- Real TSS module:
tss-lib-privateisn't available to me, so I ran theethereumpackage tests, with-raceon the new tests, against a local stub of it that isn't in this branch. Everything TSS-related did not run, so CI needs to confirm that. make lint-ciandmake generate: neither runs in full without that module.golangci-lintis clean on theethereumpackage against the stub, and this change touches no generated code.- Failed vault txs and vault txs to other contracts on live blocks: none appeared in either window, so these are covered by unit tests only.
- Direct ERC20 inbounds on live blocks: the comparison didn't count them separately. Any that appeared are among the observations through other contracts, so the unit tests are the only direct coverage.
- A provider misreporting a sender: it could hide that tx from this check, but it could equally leave the tx out of the block or misreport its receipt.
- Non-standard tokens: direct ERC20 transfers are detected from calldata, while
ParseDirectERC20TransferLogsreads logs. A token whoseTransferlog names a different recipient than its calldata could be skipped. - A vault outside
PubKeyManager: the receipts path clears the signer cache for any failed tx. A failed tx signed by a vault no longer inPubKeyManagerwould now keep its cache entry if its block is skipped. - Scan speed: I haven't measured the effect on real scan rates, so this MR claims fewer calls, not a specific speedup.