Add RateLimiting

This introduces a new rate limiting middleware for limiting request throughput. We introduce a feature flag, which if turned on will reject requests, but if turned off will not actually reject any requests, but just emits a metric for when it expects to in the future.

This change builds on the concurrency limiter. Since there is alot of shared logic (eg: when you see a request of this RPC method, limit the request under these conditions), a refactor has made it easier to add a second limiter type.

The actual rate limiter implementation is pretty straightforward and uses the https://pkg.go.dev/golang.org/x/time/rate package. The way this rate limiter works is through a concept of a token bucket. A token bucket has a capacity of Burst, and is refilled to that capacity with an interval of Interval. Once the bucket is depleted, no requests can be served until it is refilled.

We also don't want a monotonically increasing set of these rate limiters, since we need one per rpc/repo pair. For certain rpc/repo pairs that are only used once and rarely used again, we end up keeping the memory around forever. To address this, we can use a background process that prunes these rate limiters every 5 minutes.

fixes: #4026 (closed)