Resolve #3345: keep DOGE mocknet chain alive and stop duplicate DualLP deposits

Resolve #3345 (closed)

This MR implements the changes described in issue #3345 (closed).

Summary

Root causes

Job 16353814409 (acquireUser: "user has no L1 balance")

• The dogecoin container restarted mid-run. In ps.log it is up 6 minutes while its peers are up 12. The dogecoin log shows EOF reached followed by a second batch of initial blocks.
• The mocknet entrypoint ran under set -e. A single generatetoaddress call that hit dogecoind's 30s HTTP timeout exited PID 1.
• Docker then SIGKILLed dogecoind before it flushed the regtest chain. The restarted script mined a new 1000-block branch, which dropped every user's DOGE funding.
• duck in the log was an artifact: the actor logger kept appending the name of each user it checked.

Job 16347608704 (late deposit, verifyLP timeout)

• The same daemon was overloaded. Every listunspent call timed out at 5s. acquireUser queried DOGE for every user on every attempt, which used about 60s of the 2-minute budget.
• The DOGE deposit sendrawtransaction timed out, but the node accepted it anyway. The retried op then signed a second deposit from other inputs.
• THORChain added both deposits at the same height. The second one left the LP position with pending_asset set, so it never counted as complete.
• A longer time limit would not have helped, so the limits are unchanged.

Changes

• dogecoin-entrypoint.sh now supervises dogecoind:
  • Block production failures are logged and retried, with a bounded CLI timeout.
  • SIGTERM is forwarded so dogecoind flushes on stop (compose stop_grace_period: 1m).
  • After a restart, only the missing initial blocks are mined.
  • More RPC threads and a longer server timeout.
• Simulation UTXO BroadcastTx: on transport failures it re-sends the same signed payload and looks up the txid to resolve each failure. It returns ErrSimTxUncertain with the txid instead of letting the caller sign a duplicate deposit.
• DualLP acquireUser:
  • THORChain checks run first, then L1 balances are queried in RUNE order until the first funded user.
  • A user is locked only while its own L1 balance is queried.
  • The search stops after two L1 errors. That bounds a shared-daemon outage without letting one broken per-user backend (e.g. XMR) hide the other funded users.
  • When no user is found, it logs one summary that counts each reason users were skipped.
• verifyLP logs each change in the pending LP state and the time since the L1 deposit.
• Errors and timeouts name the failed step. Actor timeout logs report the op, the time spent in that op, the total elapsed time and the timeout.

Testing

• make test passes.
• make lint passes, with two exceptions that are environmental or pre-existing, not from this change:
  • The gofumpt step flags two files that already fail on develop: Tron.go and tss_test.go.
  • Trunk's golangci-lint cannot load the private tss-lib module in its sandboxed GOPATH. Run directly with --new-from-rev develop, golangci-lint reports no issues in the changed simulation packages.
• New unit tests cover:
  • DualLP candidate selection: RUNE order, L1 error budget, a failing single user, a user taken concurrently, unfunded users.
  • The LP wait state.
  • BroadcastTx timeout, accepted, rejected and uncertain cases.
  • Actor timeout fields.
• Entrypoint stub tests cover restart, EOF retry and SIGTERM.
• Run against a real dogecoin-daemon-1.14.9 container:
  • The env-gated TestDogeStalledBroadcastIsNotDuplicated passes. It stalls sendrawtransaction in both the accepted-but-unanswered and dropped cases, and the recipient is paid exactly once.
  • A container restart keeps the chain: height 5130 before and after, with no new initial branch.
• A full test-simulation run could not be done in this sandbox (3GB memory limit). This MR's pipeline will run it.

Follow-up

• litecoin-entrypoint.sh also does not forward SIGTERM to litecoind. It does not use set -e, so it does not have this crash loop. It is left out of scope here.

🤖 Generated with Claude Code