feat(bifrost): chain provider multi-host support (issue #2827)

🔎 Visual review of 0fac1c99: explainer, review flags and prompts to ask an AI about the change

Summary

Adds multi-endpoint configuration and automatic failover to bifrost chain clients (closes #2827 and #2828). Each chain takes an ordered list of full-node endpoints (self-hosted or external, same native protocol). A per-chain health monitor tracks every endpoint (HEALTHY / UNHEALTHY / RECOVERING / DISABLED), routes traffic to the active endpoint, retries reads across healthy endpoints, fails over when the active endpoint degrades, fails back when prefer_higher_priority is set, verifies that all endpoints are on the same network, and exports Prometheus metrics plus a live /status/provider view.

Existing configs are unaffected: without an endpoints list, the legacy rpc_host / api_host / cosmos_grpc_host fields form a single endpoint and the monitor simply reports on it.

Failover coverage

  • UTXO (BTC, LTC, BCH, DOGE, ZEC): reads fail over across endpoints; wallet RPC, broadcasts and signer-cache verification stay on the priority-1 daemon (it holds the wallet).
  • EVM (ETH, AVAX, BSC, BASE, POL), Solana, XRP, TRON: every call routed to the active endpoint at runtime through an http.RoundTripper; the signing chain id is the verified reference identity.
  • Cosmos (GAIA, NOBLE): gRPC (grpc.ClientConnInterface) and CometBFT RPC routed to the same active endpoint.
  • XMR: not covered (the Monero client talks to its signer sidecar only).

Safety properties

  • Writes (eth_sendRawTransaction, sendTransaction, submit, BroadcastTx, /wallet/broadcasttransaction, sendrawtransaction) are never retried on a second endpoint once they may have been sent.
  • Signer-cache verification never trusts a single endpoint that may not have seen the broadcast: UTXO asks the primary only, EVM asks every endpoint, Cosmos asks every endpoint's mempool; an unreachable endpoint makes the answer ambiguous (no re-sign).
  • Endpoints on a different network (chain id / genesis / network id) are disabled; the priority-1 endpoint disagreeing with expected_identity fails startup.
  • Lag eviction needs a quorum of two agreeing endpoints (or, with a single endpoint, the scanner's own height); a single reporter can never evict the others.
  • Client libraries are constructed against a synthetic base URL so endpoint URLs and credentials never reach *url.Error messages, logs, metric labels or status output.
  • Rate limits (429 / ResourceExhausted / JSON-RPC -32005) put an endpoint in cooldown instead of marking it unhealthy.

Deviations from the issue text

  • providers: with a type field and REST adapters became endpoints: (starsquid review); every endpoint speaks the chain's native protocol.
  • Latency: latest probe round trip only, no rolling average; no per-request latency gating.
  • Data integrity: block-hash comparison at integrity_check_depth plus identity verification; no per-transaction cross-checks.
  • Admin API at /status/provider[/{chain}] on the health port; metrics are bifrost_endpoint_*.

Test plan

  • go test -tags mocknet ./config/... ./bifrost/pkg/chainclients/... ./cmd/bifrost/...
  • make lint-ci
  • mocknet: BTC and ETH with two endpoints, stop the primary daemon, observe failover in /status/provider and bifrost_endpoint_failovers_total, restart it, observe recovery

Closes #2827 Closes #2828

Edited by Huginn Ai

Merge request reports

Loading
Loading