Ocamlnet Shell.call takes a VERY long time when max no of file descriptors is large
Shell.(call [cmd "cat" ["/etc/passwd"]]) can take minutes to execute when the max number of file descriptors is large (100,000 or 1,000,000) as is the case by default on our large FreeBSD servers.
ulimit -n 1024:
ocaml debug.ml > /dev/null 0.20s user 0.04s system 99% cpu 0.240 total
With the FreeBSD default (3772809 on this smaller server):
ocaml debug.ml > /dev/null 1.14s user 97.55s system 100% cpu 1:38.65 total
Tracing the slow-running executable shows thousands of dup2's fcntls and closes in groups like this:
24442 debug.native RET close 0 24442 debug.native CALL dup2(0x3,0x37733c) 24442 debug.native RET dup2 3633980/0x37733c 24442 debug.native CALL fcntl(0x37733c,F_SETFD,0) 24442 debug.native RET fcntl 0 24442 debug.native CALL close(0x37733c)
The problem doesn't occur for
Sys.command "cat /etc/passwd" or
Unix.execvp "cat" [|"cat";"/etc/passwd"|], or
Unix.open_process_in. All of those run equally fast no matter how many file descriptors, but of course they don't have the nice high level abstraction that Shell provides.