Instrumenting Redis connection pools in Gitlab Rails
Problem
Connection pools are a crucial component to the use of Redis in GitLab Rails. For concurrency/contention-related issues, having insight into the behaviour of the Redis connection pool could be very useful in understanding if there are any resource contention in the connection pool level.
Even if the connection pools do not timeout, having to wait < 1s is still considered slow as Redis operations are at the micro-milli second scale.
See an example issue where having such insights #2564 (comment 1668930638) could be useful.
How to instrument
Borrowing the existing pattern from database_sample may be insufficient as external gems like Sidekiq may internally hold connection pools.
We may be better off instrumenting the ConnectionPool
instance directly for metrics like .size
and .available
.