馃尶 Gitaly Cluster: Distribute Git read operations across Gitaly nodes
<!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION -->
*This page may contain information related to upcoming products, features and functionality.
It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes.
Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.*
<!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION -->
## Problem to solve
Repositories are serviced by the primary Gitaly node in the HA cluster. This means a repository with a heavy read/write load could be a noisy neighbor to all other repositories for which that Gitaly node is also primary, degrading their performance.
## Proposal
When Praefect is responding to read operation for a repository, use any Gitaly node in the cluster that is up to date.
A naive approach would be to randomly distribute the read load, but if a node is experiencing a heavy write load, this could increase the load in the worst case. However, since reads typically outnumber writes significantly, this might be a good enough first iteration?
## Links / references
epic