[bifrost] Keygen Retries and ChurnRetryInterval Mimir

Resolves #1585 (closed) - will separate out the seed refresh component noted on a comment there into a separate issue after this since it will also require more changes in go-tss. This is just a draft and has seen no testing, just pushing for now in case someone wants to carry the torch, otherwise will continue with this testing next week.

This has been tested with paths noted in comment below. Also this was extended to read a mimir for the ChurnRetryInterval.

(I echo the earlier thought that testing should ideally include whether a Ready node is able to (/fully) unbond after a keygen failure and then become an Active node on a keygen retry.)

added 19 commits

• e3144da4...5672046d - 18 commits from branch develop
• 7d354f24 - [bifrost] Keygen Retries and ChurnRetryInterval Mimir

Compare with previous version

Mocknet Testing:

• No keygen retry configured in mimir
  • Failed with one offline node, TssPool failures reported as expected
  • Offline node restarted, succeeded on next churn retry
• Keygen retry configured
  • Failed with one offline node, retried until within interval blocks of next churn retry, TssPool failures reported after retries exhausted
  • Failed with one offline node, retried until offline node enabled, next retry was successful, TssPool success reported
  • When offline node came back online and there were 2 keygen blocks since last online it still tries the first one (old keygen block with other nodes moved on), but this is consistent with current behavior and should not cause new unintended side effects

marked this merge request as ready

changed title from Draft: [bifrost] Keygen Retries to [bifrost] Keygen Retries and ChurnRetryInterval Mimir

changed the description

Would appreciate @son-of-odin / @heimdallthor review in case of any gotchas.

added 3 commits

• 8113fb44 - 1 commit from branch develop
• 687a72be - [bifrost] Keygen Retries and ChurnRetryInterval Mimir
• 0e0ae1be - heimdall feedback

Compare with previous version

added 1 commit

• 44ee8fcb - heimdall feedback

Compare with previous version

changed milestone to %Release-1.121.0

added 1 commit

• 0731113c - fix smoke

Compare with previous version

added 5 commits

• 0731113c...5d3959e6 - 2 commits from branch develop
• 0ed36be6 - [bifrost] Keygen Retries and ChurnRetryInterval Mimir
• 6fe9e397 - heimdall feedback
• 2bba6939 - fix smoke

Compare with previous version

added 4 commits

• f64823bf - [bifrost] Keygen Retries and ChurnRetryInterval Mimir
• b6816d90 - heimdall feedback
• 3b9ba529 - fix smoke
• 080138d9 - multi feedback

Compare with previous version

be aware of the slashing that occurs, which is 6 hours of yield (i think). If you're doing retry, then that slashing amount should prob be reduced to match the retry interval

added 9 commits

• 080138d9...129d15a5 - 4 commits from branch develop
• a747f12e - [bifrost] Keygen Retries and ChurnRetryInterval Mimir
• d87efbf4 - heimdall feedback
• 8784e074 - fix smoke
• 48cbffbf - multi feedback
• 904e4410 - add retry to keygen broadcast, remove dead backoff config

Compare with previous version

added 1 commit

• 48b07306 - add retry to keygen broadcast, remove dead backoff config

Compare with previous version