There are multiple issues with the code which searches for the caught-up replica in LoadBalancer: 1. It walks over all replicas to find only a single one that is caught up. And also it would fall back to primary even not all of the replicas are caught up. 2. `VALID_HOSTS_CACHE_KEY` logic operates under wrong assumptions and doesn't work as intended: https://gitlab.com/groups/gitlab-org/-/epics/6320#note_622449455 3. The `return unless location` logic (multiple places) We propose to resolve it in steps: 1. We would like to have an **additional metric**. It would show us how often are we not able to unstick. Assuming, that it happens because of the pretty strict `load_balancer.all_caught_up?(location)` check which currently requires all replicas being caught up. With the current behavior, having one lagging replica would disallow us using caught-up replicas. We consider using `ActiveSupport::Notifications` with the custom subscriber. We may want to fire an event in `Sticking.all_caught_up?`. :white_check_mark: 1. After we'll release the metric, we'll get an idea of how often do we unable to use replicas. We'll change the strategy of `all_caught_up?`, so it would require having at least one replica to be caught up to proceed with unsticking/not forcing a primary. It'll be under the feature flag for safe rollout. :white_check_mark: 1. Rework retry logic in `read`: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/65755#note_622034549 1. Then there's another story with how we return early in case the `location` is not set which Heinrich mentioned. I feel like it is a different problem, even though it sits in the same method. Maybe we may even want to split it into a separate issue. I consider this independent of the "checking every replica" problem.