Set GitlabCI job priority mechanism
Problem
As we are maturing our usage of Gitlab and its API to automate code changes through MRs across large amount of repositories, we are running into issues regarding job queue congestion. While we will be looking at spreading MRs over time to avoid a spike in jobs, but this would also artificially delay getting feedback from the process. We expect that in the future we won't be able to have a central control over these automated changes as more security tools are implementing these correction mechanisms.
Proposal
We would like to have a way to set a job priority based on certain conditions (user, commit message, etc), so that in the event that they are too many jobs to be processed, the lower-priority ones don't create delay for the normal jobs our users expect to be processed in a timely manner. This would be akin to giving a UI thread a higher CPU priority than a background one, so that the user experience isn't degraded while waiting for a task completion.
While this has some similarities with gitlab-runner#1873 (closed), this is different because the priority would be set outside of the .gitlab-ci.yml definition, probably based on the (automated) user pushing commits, creating Merge Requests or triggering pipelines through the API.