Follow up: Improve finding caught up replica for SidekiqServerMiddleware
-
@reprazent started a discussion
Would it be worth creating a new
#select_up_to_date_host
that shuffles and queries the replica list for the first up to date one? That would reduce some of the dependency we have on sidekiq-retries for these jobs, since it's less likely for none of the replicas to have caught up. WDYT?
In !49294 (merged) we intorduced LoadBalancer#select_caught_up_hosts
.
The idea is that we could use a similar approach for SidekiqServerMiddleware
.
We don't want to query all replicas for each job, but we could create a new #select_up_to_date_host
that shuffles and queries the replica list for the first up-to-date
one?
This will ensure that:
- we don't end up using only one replica.
- it is more likely to find a replica that has caught up, so we don't need to retry, or fall-back to the primary
Edited by Nikola Milojevic