feat(bifrost): chain provider multi-host support (issue #2827)
🔎 Visual review of0fac1c99: explainer, review flags and prompts to ask an AI about the change
Summary
Adds multi-endpoint configuration and automatic failover to bifrost chain clients
(closes #2827 and #2828). Each chain takes an ordered list of full-node endpoints
(self-hosted or external, same native protocol). A per-chain health monitor tracks
every endpoint (HEALTHY / UNHEALTHY / RECOVERING / DISABLED), routes traffic to the
active endpoint, retries reads across healthy endpoints, fails over when the active
endpoint degrades, fails back when prefer_higher_priority is set, verifies that all
endpoints are on the same network, and exports Prometheus metrics plus a live
/status/provider view.
Existing configs are unaffected: without an endpoints list, the legacy rpc_host /
api_host / cosmos_grpc_host fields form a single endpoint and the monitor simply
reports on it.
Failover coverage
- UTXO (BTC, LTC, BCH, DOGE, ZEC): reads fail over across endpoints; wallet RPC, broadcasts and signer-cache verification stay on the priority-1 daemon (it holds the wallet).
- EVM (ETH, AVAX, BSC, BASE, POL), Solana, XRP, TRON: every call routed to the active
endpoint at runtime through an
http.RoundTripper; the signing chain id is the verified reference identity. - Cosmos (GAIA, NOBLE): gRPC (
grpc.ClientConnInterface) and CometBFT RPC routed to the same active endpoint. - XMR: not covered (the Monero client talks to its signer sidecar only).
Safety properties
- Writes (
eth_sendRawTransaction,sendTransaction,submit,BroadcastTx,/wallet/broadcasttransaction,sendrawtransaction) are never retried on a second endpoint once they may have been sent. - Signer-cache verification never trusts a single endpoint that may not have seen the broadcast: UTXO asks the primary only, EVM asks every endpoint, Cosmos asks every endpoint's mempool; an unreachable endpoint makes the answer ambiguous (no re-sign).
- Endpoints on a different network (chain id / genesis / network id) are disabled;
the priority-1 endpoint disagreeing with
expected_identityfails startup. - Lag eviction needs a quorum of two agreeing endpoints (or, with a single endpoint, the scanner's own height); a single reporter can never evict the others.
- Client libraries are constructed against a synthetic base URL so endpoint URLs and
credentials never reach
*url.Errormessages, logs, metric labels or status output. - Rate limits (429 /
ResourceExhausted/ JSON-RPC -32005) put an endpoint in cooldown instead of marking it unhealthy.
Deviations from the issue text
providers:with atypefield and REST adapters becameendpoints:(starsquid review); every endpoint speaks the chain's native protocol.- Latency: latest probe round trip only, no rolling average; no per-request latency gating.
- Data integrity: block-hash comparison at
integrity_check_depthplus identity verification; no per-transaction cross-checks. - Admin API at
/status/provider[/{chain}]on the health port; metrics arebifrost_endpoint_*.
Test plan
-
go test -tags mocknet ./config/... ./bifrost/pkg/chainclients/... ./cmd/bifrost/... -
make lint-ci - mocknet: BTC and ETH with two endpoints, stop the primary daemon, observe failover in
/status/providerandbifrost_endpoint_failovers_total, restart it, observe recovery