Git clone failures when building many systems in GitLab CI concurrently
I have a branch doing BuildStream conversions of Baserock systems, and it has a CI pipeline which builds about 10 systems in parallel.
These jobs keep failing with issues like this:
/usr/bin/git clone --mirror -n git://git.baserock.org/delta/python-packages/beautifulsoup4.git /cache/buildstream/sources/git/tmpe1poqrhe
Cloning into bare repository '/cache/buildstream/sources/git/tmpe1poqrhe'...
fatal: read error: Connection reset by peer
[00:00:06] FAILURE [unmaintained/test-tools/beautifulsoup4.bst-0]: Fetching git://git.baserock.org/delta/python-packages/beautifulsoup4.git
[00:00:06] FAILURE [unmaintained/test-tools/beautifulsoup4.bst]: Fetch
On git.baserock.org, I see messages like this in journalctl --unit=git-daemon.service
which match the timings of the failing jobs:
Aug 17 11:24:06 firehose1 git[6087]: [6089] Too many children, dropping connection
Seems we have some sort of limit that should perhaps be increased.