[Version-unspecific] feat(bifrost): expire signer cache entries and re-sign dropped outbounds

Problem

After a node broadcasts an outbound, it records the outbound in its signer cache. That entry never expired and was never checked against the chain. If the broadcast later left the mempool (eviction, node restart without a persisted mempool, a conflicting reorg), every later SignTx for that outbound hit HasSigned and returned without signing. That includes MAYAChain's reschedule, even to another vault, because CacheHash holds no vault and no height. This node stayed out of the keysign until an operator cleared the cache by hand. No UTXO client ever calls RemoveSigned.

This ports THOR thornode!4770 ("Signer Cache Expiry for All Chains", a759cb4f99) together with the follow-ups from thornode!4985 (9f17b66f8d): the UTXO canonical-chain check and the CometBFT 100-tx page cap.

Change

Signer cache (shared/signercache)

  • New SetSignedWithTTL / HasSignedWithTTL. They store a wall-clock expiry (Unix ms) under a new ttl-v6- key, separate from the expiry-v6- height that ZEC uses. The write is one synced batch, the same as ZEC's SetSignedWithExpiry.

  • RemoveSigned, RemoveSignedByCacheHash and ClearSignedTransaction also drop the TTL key. SetSigned (the observation path) keeps it when the observed tx is the recorded broadcast, and drops it for any other tx.

  • Entries without a TTL never expire. This covers legacy entries, outbounds first recorded by an observation of another tx (SetSigned) and ZEC.

  • New SignTxCacheCheck(cacheHash, gateFn, verifyFn):

    State Action
    No entry sign
    Entry not expired skip
    Expired, gate closed skip
    Expired, no recorded tx id skip
    Recorded tx found on chain skip
    Chain reply ambiguous skip
    Recorded tx absent on chain sign; the entry stays until the new broadcast replaces it, so a failed sign is re-checked

    The gate is read lazily, so a fresh or missing entry costs no mimir query.

  • SkipSignerCacheGate reads the SkipSignerCache<CHAIN> mimir through the existing GetMimirWithRef bridge call. A read error keeps the gate closed.

Config

  • New field bifrost.chains.<chain>.signer_cache_expiry, default 72h, set on the default-chain anchor.
  • A value of zero or less falls back to 72h.

Constants

  • New MimirTemplateSkipSignerCache = "SkipSignerCache%s". Only bifrost reads it. The state machine does not use it.

Chain clients

Chain Client On-chain check
BTC bitcoin/ getrawtransaction verbose. A tx whose block was orphaned (0 confirmations, block hash set) counts as absent. On not-found it re-checks the mempool, as confirmTx does. Uses reorg.IsUTXOTxNotFoundErr, so a "use -txindex" reply is ambiguous.
DASH dash/ getrawtransaction verbose. The dashd-go RPC error goes through the same classifier.
ETH ethereum/ TransactionByHash, which also sees pending txs. Only ethereum.NotFound counts as absent. Unstuck cancel txs are cached with the TTL.
ARB, AVAX evm/ Same as ETH.
THOR thorchain/ Reads one page of the CometBFT mempool (100 txs) first, then the latest height h from Status, then gRPC GetTx. A NotFound counts as absent only if the mempool page was read in full without the tx and BlockSearch shows block h+1 indexed (waits up to 15s; otherwise ambiguous). Uses the scanner's RPC client.
KUJI kuji/ Same as THOR.
ADA cardano/ The cardano-rpc sidecar's 404 transaction not found. During a backfill the sidecar answers not_yet_indexed, which is ambiguous.

Unchanged

Chain Reason
ZEC Already reconciles cache entries against the tx expiry height plus a reorg margin (tx_lifecycle.go).
XRD Already checks the intent status on every SignTx (shouldClearStaleCache).
BNB Disabled in config. The chain is retired.

Operating it

  • Nothing changes until an entry is 72h old and SkipSignerCache<CHAIN> is set to a positive value. The chain token is the chain id, for example SkipSignerCacheBTC.
  • Admin can set this mimir. It is not on the mainnet admin deny list.
  • Nodes can also set it, but as an economic mimir that needs a supermajority. It becomes operational from 1.136.0 (V136 stack).
  • Every node reads the same consensus value, so all bifrosts take the same decision.
  • Unset it once the stuck outbound has gone out.

Consensus impact

None.

  • No handler, keeper, manager or constant value changed.
  • The new mimir key is only read by bifrost.
  • THOR also adds SKIPSIGNERCACHE to IsOperationalMimir, which lets a plurality vote set it. In MAYA that would be a consensus change: isOperationalMimirV135 is version-gated, so it would need a new V136 variant. It is left out of this version-unspecific MR and added on the V136 stack instead: isOperationalMimirV136 on branch v136/00-chain-activation (6f331b62b) makes SkipSignerCache<chain> operational from 1.136.0.

THORNode references and gap check

  • thornode!4770 "Signer Cache Expiry for All Chains" (a759cb4f99).
  • thornode!4985 "!4770 review follow-ups" (9f17b66f8d).
  • Every change in both MRs was compared with this port. All changes for chains MAYA has on develop are ported or adapted: signer cache helper, config default, mimir key, UTXO canonical check, CometBFT mempool lookup with the 100-tx cap, ETH/EVM lookup, tests.
  • Not ported:
    • SKIPSIGNERCACHE in IsOperationalMimir. It is a consensus change, so it is on the V136 stack (v136/00-chain-activation, 6f331b62b); see "Consensus impact".
    • The XRP, TRON and SOL wiring. MAYA has no XRP on develop; TRON and SOL are on the V136 branches.
    • THOR's removal of the 0 fallback. MAYA keeps "0 or less means 72h", which is safer than THOR's immediate expiry.
  • docs/mimir.md now documents SkipSignerCache<chain>. THOR does not document it.
  • No later THOR change touches this code on develop. The SOL-only follow-ups thornode!5041 and thornode!5128 matter only for the SOL port.

Tests

  • shared/signercache/sign_tx_check_test.go:
    • every phase of SignTxCacheCheck;
    • the gate is not read for a fresh entry;
    • entries without a TTL never expire;
    • SetSigned keeps the TTL for the recorded broadcast (also after a leveldb reopen) and drops it for another tx; RemoveSigned drops it;
    • a malformed TTL is treated as not expired;
    • ResolveTTL, UTXOPresence and SkipSignerCacheGate.
  • verify_broadcast_test.go in bitcoin, dash, ethereum, evm, thorchain, kuji and cardano maps each chain's replies to found / not found / ambiguous:
    • BTC and DASH use a JSON-RPC stub covering confirmed, mempool, orphaned, -5 not-found, -5 with the "-txindex" hint, and RPC down. BTC also covers not-found but still in the mempool.
    • ETH and EVM use a stub covering present, null result, JSON-RPC error and HTTP 500.
    • THOR and KUJI use a fake CometBFT node (mempool, Status, BlockSearch, GetTx) covering committed, pending, a commit during the mempool read, committed but not indexed, absent with the indexer past the latest block, block indexing disabled, a Status error, a mempool larger than one page, a mempool error and gRPC unavailable.
    • ADA covers found, 404, not_yet_indexed, RPC failure and a transport error.
  • Command: go test -tags=mocknet -p 2 -count=1 on all touched packages. All pass.

Follow-ups

  • A2 (THOR cc8097283b): EVM/ETH unstuck does not clear the signer cache when an outbound was dropped or reorged out. This MR does not change that path.
  • Once the V136 chain stack lands, the new clients need the same wiring:
    • bitcoincash/ (BCH): SignTxCacheCheck with UTXOPresence + reorg.IsUTXOTxNotFoundErr, and SetSignedWithTTL in BroadcastTx;
    • TRON and SOL: their THOR clients already use this pattern, with signer_cache_expiry: 2h because their keysigns embed a recent block;
    • BSC: covered through evm/.

Review follow-up (round 4)

  • 9dd94f47a: SetSigned keeps the TTL when the node observes its own broadcast (same tx id, ignoring 0x and hex case), in one locked batch. Another tx still makes the entry permanent.
  • bb0fa062b: a definitive absence no longer clears the entry before signing (PhaseVerifiedAbsent). Retries re-check the Mimir and the chain, and SetSignedWithTTL drops the old tx id's map when the re-signed broadcast replaces the entry.
  • 6c856206b: THOR and KUJI verifyBroadcast reads the mempool first and counts a GetTx NotFound only once the CometBFT indexer has passed the latest block (BlockSearch of h+1, bounded at 15s).
  • 190db19f9: checked type assertion in a test (lint).
Edited by Bitol Maya

Merge request reports

Loading
Loading