Add configurable "db_pool_headroom" for the connection pool in multithreaded environments
To avoid issues like the ones we've seen in https://gitlab.com/gitlab-com/gl-infra/infrastructure/issues/9198 we could make the connection pool bigger than we actually need.
Normally, every thread gets it's own connection, but in the issue linked above, more threads were started then we anticipated. The problem didn't surface before because one of these threads didn't connect to the database. When gitlab-org/gitlab!24388 (merged) was deployed it did. Causing jobs to fail especially with low concurrency.
To avoid this problem in the future, we should add some headroom to the connection pool. We can do this since our connections go to pgbouncer and not directly to the database. Which means they are cheap. But not every environment runs like that, so we need to make that variable configurable in omnibus and the charts.