Separate CI queue time from CI job timeout
Currently there is a time out value for the entire length of a job. This will use both queuing time (waiting for a runner) as well as job execution time.
This means that if I have a job time out of 60 minutes and the job is waiting for the runner for 59 minutes, it will be timed out 1 minute into execution. Or if it is waiting longer than 60 minutes than it will be timed out before it even starts.
I have a single CI runner that builds many projects. However since a few months, the projects in the pipeline often time out before ever getting started. I assume that the project build time of 3600 seconds is applied also as a queue waiting time? If this is the case, in my understanding they are fundamentally different things. Every project in the build queue should at least get started before it fails. It seems wrong that projects "fail" building just because runners are busy.
Add a global setting to configure CI queuing time.
Links / references
Possibly related to https://gitlab.com/gitlab-org/gitlab-ce/issues/22081