Stop replica catch-up check at first caught up host

What does this merge request do and why?

LoadBalancer#select_up_to_date_host runs caught_up? on every replica. That is one replica query for each host on each sticky or delayed Sidekiq job. Only Sticking#find_caught_up_replica needs to know if all hosts are caught up, so that it can clear the write location in Redis. WalTrackingReceiver#databases_in_sync? (used by SidekiqServerMiddleware and ActionCableWithLoadBalancing) only compares the result with NONE_CAUGHT_UP, so one caught up host is enough.

This adds an all_hosts: keyword that defaults to true, so Sticking and any new caller keep the full check. WalTrackingReceiver passes all_hosts: false and stops at the first caught up host in the same shuffled order, so the chosen host, the host spread, the request cache value and the caught_up_replica_pick.load_balancing event do not change. The only change for the receiver is that the result is :any instead of :all when every host is caught up, and the receiver does not read :all. The new path is behind the gitlab_com_derisk flag load_balancer_stop_at_first_caught_up_host (default off, Feature.current_request actor).

References

Screenshots or screen recordings

Before After

How to set up and validate locally

  1. Enable the flag:
    Feature.enable(:load_balancer_stop_at_first_caught_up_host)
  2. Run bin/rspec spec/lib/gitlab/database/load_balancing/load_balancer_spec.rb.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Phil Hughes

Merge request reports

Loading
Loading