Extend retry-period for read-only Gitaly RPCs
What does this MR do and why?
Extend retry-period for read-only Gitaly RPCs
With !133896 (merged) we configured the Gitaly client to send up to two retries on read-only RPCs, waiting no more than 750ms in total.
After rolling out the new Gitaly upgrade process to canary, we have observed that Gitaly restarts may take up to two seconds to begin accepting traffic, more than the grace period configured for retries.
Expand the retry policy to allow up to three retries with a starting delay of 0.33 seconds and a maximum backoff of 1.5 seconds. This gives us a total grace period of 2.3 seconds, which is still well within the default deadline of 10 seconds.
MR acceptance checklist
Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
How to set up and validate locally
- Stop your GDK's Gitaly with
gdk stop gitaly
- Load a rails console session and execute a
FindCommit
withProject.last.repository.commit('HEAD')
- Immediately start Gitaly with
gdk start gitaly
. If Gitaly starts fast enough, the request will succeed with output like=> #<Commit id:603975295665c2601289682bd3eefe92da22f848 i-user-0-1696879720/lab-coat@603975295665c2601289682bd3eefe92da22f848>
- If you have trouble getting the timing right, increasing
maxAttempts
andmaxBackoff
may help.
- If you have trouble getting the timing right, increasing