fix: pgbouncer_client_conn saturation flapping
What
Increase the saturation soft and hard SLO for the
pgbouncer_client_conn
from 80% (soft) and 85% (hard) to 90% (soft) to
95% (hard)
Why
In https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/16121 we
are working on increasing the max_client_conn
for the primary
pgbouncers, however, this requires testing and a better understanding if
we saturating CPU or if there would be a knock-on effect on the Primary
CPU usage. In the mean time, the EoC is getting paged 2-3 times a
day
with no action to take.
We don't want to create silence for this alert because this will have negative effect to our users if we are 100% saturated so the EoC should still, be aware that we are saturating if this happens.
Reference: https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/16121