[Version-unspecific] feat(bifrost): expire signer cache entries and re-sign dropped outbounds
Problem
After a node broadcasts an outbound, it records the outbound in its signer cache. That entry never expired and was never checked against the chain. If the broadcast later left the mempool (eviction, node restart without a persisted mempool, a conflicting reorg), every later SignTx for that outbound hit HasSigned and returned without signing. That includes MAYAChain's reschedule, even to another vault, because CacheHash holds no vault and no height. This node stayed out of the keysign until an operator cleared the cache by hand. No UTXO client ever calls RemoveSigned.
This ports THOR thornode!4770 ("Signer Cache Expiry for All Chains", a759cb4f99) together with the follow-ups from thornode!4985 (9f17b66f8d): the UTXO canonical-chain check and the CometBFT 100-tx page cap.
Change
Signer cache (shared/signercache)
-
New
SetSignedWithTTL/HasSignedWithTTL. They store a wall-clock expiry (Unix ms) under a newttl-v6-key, separate from theexpiry-v6-height that ZEC uses. The write is one synced batch, the same as ZEC'sSetSignedWithExpiry. -
RemoveSigned,RemoveSignedByCacheHashandClearSignedTransactionalso drop the TTL key.SetSigned(the observation path) keeps it when the observed tx is the recorded broadcast, and drops it for any other tx. -
Entries without a TTL never expire. This covers legacy entries, outbounds first recorded by an observation of another tx (
SetSigned) and ZEC. -
New
SignTxCacheCheck(cacheHash, gateFn, verifyFn):State Action No entry sign Entry not expired skip Expired, gate closed skip Expired, no recorded tx id skip Recorded tx found on chain skip Chain reply ambiguous skip Recorded tx absent on chain sign; the entry stays until the new broadcast replaces it, so a failed sign is re-checked The gate is read lazily, so a fresh or missing entry costs no mimir query.
-
SkipSignerCacheGatereads theSkipSignerCache<CHAIN>mimir through the existingGetMimirWithRefbridge call. A read error keeps the gate closed.
Config
- New field
bifrost.chains.<chain>.signer_cache_expiry, default72h, set on thedefault-chainanchor. - A value of zero or less falls back to 72h.
Constants
- New
MimirTemplateSkipSignerCache = "SkipSignerCache%s". Only bifrost reads it. The state machine does not use it.
Chain clients
| Chain | Client | On-chain check |
|---|---|---|
| BTC | bitcoin/ |
getrawtransaction verbose. A tx whose block was orphaned (0 confirmations, block hash set) counts as absent. On not-found it re-checks the mempool, as confirmTx does. Uses reorg.IsUTXOTxNotFoundErr, so a "use -txindex" reply is ambiguous. |
| DASH | dash/ |
getrawtransaction verbose. The dashd-go RPC error goes through the same classifier. |
| ETH | ethereum/ |
TransactionByHash, which also sees pending txs. Only ethereum.NotFound counts as absent. Unstuck cancel txs are cached with the TTL. |
| ARB, AVAX | evm/ |
Same as ETH. |
| THOR | thorchain/ |
Reads one page of the CometBFT mempool (100 txs) first, then the latest height h from Status, then gRPC GetTx. A NotFound counts as absent only if the mempool page was read in full without the tx and BlockSearch shows block h+1 indexed (waits up to 15s; otherwise ambiguous). Uses the scanner's RPC client. |
| KUJI | kuji/ |
Same as THOR. |
| ADA | cardano/ |
The cardano-rpc sidecar's 404 transaction not found. During a backfill the sidecar answers not_yet_indexed, which is ambiguous. |
Unchanged
| Chain | Reason |
|---|---|
| ZEC | Already reconciles cache entries against the tx expiry height plus a reorg margin (tx_lifecycle.go). |
| XRD | Already checks the intent status on every SignTx (shouldClearStaleCache). |
| BNB | Disabled in config. The chain is retired. |
Operating it
- Nothing changes until an entry is 72h old and
SkipSignerCache<CHAIN>is set to a positive value. The chain token is the chain id, for exampleSkipSignerCacheBTC. - Admin can set this mimir. It is not on the mainnet admin deny list.
- Nodes can also set it, but as an economic mimir that needs a supermajority. It becomes operational from 1.136.0 (V136 stack).
- Every node reads the same consensus value, so all bifrosts take the same decision.
- Unset it once the stuck outbound has gone out.
Consensus impact
None.
- No handler, keeper, manager or constant value changed.
- The new mimir key is only read by bifrost.
- THOR also adds
SKIPSIGNERCACHEtoIsOperationalMimir, which lets a plurality vote set it. In MAYA that would be a consensus change:isOperationalMimirV135is version-gated, so it would need a newV136variant. It is left out of this version-unspecific MR and added on the V136 stack instead:isOperationalMimirV136on branchv136/00-chain-activation(6f331b62b) makesSkipSignerCache<chain>operational from 1.136.0.
THORNode references and gap check
- thornode!4770 "Signer Cache Expiry for All Chains" (
a759cb4f99). - thornode!4985 "!4770 review follow-ups" (
9f17b66f8d). - Every change in both MRs was compared with this port. All changes for chains MAYA has on develop are ported or adapted: signer cache helper, config default, mimir key, UTXO canonical check, CometBFT mempool lookup with the 100-tx cap, ETH/EVM lookup, tests.
- Not ported:
SKIPSIGNERCACHEinIsOperationalMimir. It is a consensus change, so it is on the V136 stack (v136/00-chain-activation,6f331b62b); see "Consensus impact".- The XRP, TRON and SOL wiring. MAYA has no XRP on develop; TRON and SOL are on the V136 branches.
- THOR's removal of the 0 fallback. MAYA keeps "0 or less means 72h", which is safer than THOR's immediate expiry.
docs/mimir.mdnow documentsSkipSignerCache<chain>. THOR does not document it.- No later THOR change touches this code on develop. The SOL-only follow-ups thornode!5041 and thornode!5128 matter only for the SOL port.
Tests
shared/signercache/sign_tx_check_test.go:- every phase of
SignTxCacheCheck; - the gate is not read for a fresh entry;
- entries without a TTL never expire;
SetSignedkeeps the TTL for the recorded broadcast (also after a leveldb reopen) and drops it for another tx;RemoveSigneddrops it;- a malformed TTL is treated as not expired;
ResolveTTL,UTXOPresenceandSkipSignerCacheGate.
- every phase of
verify_broadcast_test.goinbitcoin,dash,ethereum,evm,thorchain,kujiandcardanomaps each chain's replies to found / not found / ambiguous:- BTC and DASH use a JSON-RPC stub covering confirmed, mempool, orphaned, -5 not-found, -5 with the "-txindex" hint, and RPC down. BTC also covers not-found but still in the mempool.
- ETH and EVM use a stub covering present,
nullresult, JSON-RPC error and HTTP 500. - THOR and KUJI use a fake CometBFT node (mempool,
Status,BlockSearch,GetTx) covering committed, pending, a commit during the mempool read, committed but not indexed, absent with the indexer past the latest block, block indexing disabled, aStatuserror, a mempool larger than one page, a mempool error and gRPC unavailable. - ADA covers found, 404,
not_yet_indexed, RPC failure and a transport error.
- Command:
go test -tags=mocknet -p 2 -count=1on all touched packages. All pass.
Follow-ups
- A2 (THOR
cc8097283b): EVM/ETH unstuck does not clear the signer cache when an outbound was dropped or reorged out. This MR does not change that path. - Once the V136 chain stack lands, the new clients need the same wiring:
bitcoincash/(BCH):SignTxCacheCheckwithUTXOPresence+reorg.IsUTXOTxNotFoundErr, andSetSignedWithTTLinBroadcastTx;- TRON and SOL: their THOR clients already use this pattern, with
signer_cache_expiry: 2hbecause their keysigns embed a recent block; - BSC: covered through
evm/.
Review follow-up (round 4)
9dd94f47a:SetSignedkeeps the TTL when the node observes its own broadcast (same tx id, ignoring0xand hex case), in one locked batch. Another tx still makes the entry permanent.bb0fa062b: a definitive absence no longer clears the entry before signing (PhaseVerifiedAbsent). Retries re-check the Mimir and the chain, andSetSignedWithTTLdrops the old tx id's map when the re-signed broadcast replaces the entry.6c856206b: THOR and KUJIverifyBroadcastreads the mempool first and counts aGetTxNotFound only once the CometBFT indexer has passed the latest block (BlockSearchofh+1, bounded at 15s).190db19f9: checked type assertion in a test (lint).