Increase maxConnections or Make It Configurable for Large-Scale CI Jobs

We encountered an issue when running a large number of parallel CI jobs that utilize the watch operation. The following error occurs frequently:

GitLab Agent Server: Proxy failed to make outbound request: rpc error: code = DeadlineExceeded desc = finding tunnel timed out.

Root Cause:

This appears to be related to the maxConnections limit, which is currently set to 500. When the number of parallel connections exceeds this limit, the server cannot handle additional requests, leading to timeouts.

Reference:

A similar issue was discussed previously, and a ticket was created to increase the maxConnections limit: #378 (closed).

Suggestion:

To provide more flexibility for large-scale CI environments, it might be beneficial to:

  1. Increase the default value of maxConnections.

  2. Expose the maxConnections setting in the configuration file, allowing users to adjust it based on their specific requirements. This would help accommodate environments with higher demands while maintaining performance and stability.