High number of open file descriptors due to Unicorn TCP sampling via Raindrops
Customer observed a large number of file descriptors open since upgrading to 9.4:
A lsof
shows a large number of file descriptors used by netlink
sockets, which are used by Raindrops to probe the status of TCP sockets: https://github.com/tmm1/raindrops/blob/1c18fd9c13f95fef6bcbdc0587d38886fa8e9064/README#L29-L31
Commenting out the Gitlab::Metrics::UnicornSampler
line in https://gitlab.com/gitlab-org/gitlab-ce/blob/v9.4.5/config/initializers/8_metrics.rb#L125 appears to make this stop.
Zendesk: https://gitlab.zendesk.com/agent/tickets/81430
/cc: @pchojnacki, @bjk-gitlab, @joshlambert