Allow Gitaly to push back on traffic surges
DRI: @qmnguyen0711
To reduce the risk of incidents like [this one](https://gitlab.com/gitlab-com/gl-infra/production/-/issues/6172), Gitaly should be able to monitor its traffic and push back on its clients when it determines some limit has been reached.
The clients of Gitaly (gitlab-shell, workhorse, rails) should know how to interpret errors from Gitaly that indicate too many requests, and return a sensible error to the producer of the traffic, whether that be the application, an API call, an ssh clone, or an http clone.
This will greatly protect the Gitlab ecosystem as a whole from both bad actors as well as surges of traffic from a few select users that can affect the experience for other users.
## Updates
All short-term initiatives are done. The Remote IP concurrency limiting was fully out. In long term, we aim to build an overload detector and load-shedding strategy and can drop requests when Gitaly process: https://gitlab.com/groups/gitlab-org/-/epics/10734
* Short term:
* Rollout static limiters
* :white_check_mark: https://gitlab.com/gitlab-org/gitaly/-/issues/5012+
* :white_check_mark: https://gitlab.com/gitlab-org/gitaly/-/issues/5013+
* :white_check_mark: https://gitlab.com/gitlab-org/gitaly/-/issues/4413+
* Make clients to be cool with backpressure
* :white_check_mark: : https://gitlab.com/gitlab-org/gitaly/-/issues/5023+
* :white_check_mark: https://gitlab.com/gitlab-org/gitlab/-/issues/358824+
* :white_check_mark: https://gitlab.com/gitlab-org/gitaly/-/issues/5070+
* :white_check_mark: https://gitlab.com/gitlab-org/gitaly/-/issues/5069+
* :white_check_mark: : https://gitlab.com/gitlab-org/gitaly/-/issues/5068+
* Long term: https://gitlab.com/groups/gitlab-org/-/epics/10734+
epic