Idea to consider: Implement Redis queue for shared runners only
We do have 5 shared runners managers configured to pick builds without tags. These runners process around 400-500k builds each day.
Implement Redis queuing for untagged builds running on shared runners only. This will allow us to iteratively move towards the next version of CI/CD queuing while giving us time to build the proper solution.
Processing of builds without tags on 25.02.2021:
|Runner||All builds on 25.02||Builds without tags||Percentage|
See more data in #322972 (comment 518063713)
- Builds ordering - we can only guarantee an order per runner type (like today)
- Fair scheduling - separate priority queues possible