Stop replica catch-up check at first caught up host
What does this merge request do and why?
LoadBalancer#select_up_to_date_host runs caught_up? on every replica. That is one replica query for each host on each sticky or delayed Sidekiq job. Only Sticking#find_caught_up_replica needs to know if all hosts are caught up, so that it can clear the write location in Redis. WalTrackingReceiver#databases_in_sync? (used by SidekiqServerMiddleware and ActionCableWithLoadBalancing) only compares the result with NONE_CAUGHT_UP, so one caught up host is enough.
This adds an all_hosts: keyword that defaults to true, so Sticking and any new caller keep the full check. WalTrackingReceiver passes all_hosts: false and stops at the first caught up host in the same shuffled order, so the chosen host, the host spread, the request cache value and the caught_up_replica_pick.load_balancing event do not change. The only change for the receiver is that the result is :any instead of :all when every host is caught up, and the receiver does not read :all. The new path is behind the gitlab_com_derisk flag load_balancer_stop_at_first_caught_up_host (default off, Feature.current_request actor).
References
- Rollout issue: #638923
Screenshots or screen recordings
| Before | After |
|---|---|
How to set up and validate locally
- Enable the flag:
Feature.enable(:load_balancer_stop_at_first_caught_up_host) - Run
bin/rspec spec/lib/gitlab/database/load_balancing/load_balancer_spec.rb.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.