Prevent project to occupy complete runner fleet
We have a small runner fleet, and sometimes the one or the other project starts enough pipelines to occupy all of them, preferably with long-running jobs. We need a solution to prevent this, as all of the workarounds known do not fit the usecase:
- Dedicated Tag
With a dedicated runner tag for long-running jobs (or even for that project only) I have to rely on the developers applying that tag. Which indicates that (first) they know about this and (second) they have the patience to not just remove that tag requirement from their jobs when their job waits for some time.
- Dedicated Runner
Having a dedicated runner for just that one project seems to be a waste of time and ressources. Time because at the moment they can use 3 runners (Ok, I want them to be able to use 2 of them at the same time) and with a dedicated runner it`s only one. Ressources because this one project by far doesn't saturate a runner, so it will idle most of the time and could do other project's work.
- Resource group
Similar to "dedicated tag": One has to rely on developers to add it.
Another problem with all 3 workarounds: It is always a reaction - I have to monitor the fleet and if another project comes up occupying the fleet I have to beg the devs to add a resource_group/tag/setup another runner. (Additionally, monitoring of the runners and GitLab CI is not the easiest task with a lot of interesting metrics missing, it wasn`t even before monitoring became deprecated...)
Solution
Easiest to think of would be a global GitLab-setting "allow one project to use only x% of the runner fleet with the same tag at once" or something like that.