High number of open pack/idx files in gitaly-ruby
While investigating NFS woes for two customers (https://gitlab.com/gitlab-org/gitlab-ce/issues/52017#note_106860178), I found that gitaly-ruby keeps open file descriptors to Git pack and index files. On file-19, for example, there were over 30,000 open files, 200 of which were to deleted files:
$ sudo lsof | grep "/var/opt/gitlab/git-data/repositories" > /tmp/open_files.txt
$ cat /tmp/open_files.txt | grep deleted | wc -l
200
$ cat /tmp/open_files.txt | wc -l
35116
Since these are memory-mapped files, this also leads to wasted memory usage.
This probably happens because any open references to Rugged::Repository
don't close files until they are garbage collected.
In the past, we fixed this by calling Rugged::Repository#close
, which frees open file descriptors in libgit2.
Edited by Stan Hu