Skip to content

Introduce open_fds saturation point for process_exporter

AFAICT we have a gap in our file descriptor monitoring. While we do have an open_fds saturation point, this looks only at process_open_fds / process_max_fds. It applies only to in-process exporters that actually are able to self-report these metrics.

We have several services that are sensitive to open fd limits:

  • gitaly
  • haproxy
  • redis

For gitaly we are able to rely on the in-process metrics. For haproxy and redis however, we rely on process_exporter.

process_exporter exposes a metric that does what we want: namedprocess_namegroup_worst_fd_ratio (source).

We are not currently monitoring this metric in saturation points and subsequently capacity planning. We can see that several of our redises actually have fairly high values here:

Screenshot_2024-01-29_at_12.55.52

source

This was discovered in light of investigating #2754 (closed).