WIP: Use Redis for CI job queueing
What does this MR do?
Another take on #34401.
Currently, it is full Redis-only approach with two-level bucketing to allow spreading of the load and offer "more fair scheduling".
This is Proof of Concept which I'm quite happy of. The
jobs/request stays, but instead it becomes very cheap:
- I got rid of expensive SQL to balance jobs (this did not scale),
- Once the job is picked a normal flow of transitioning the job is done: find object, transition.
- I introduced additional worker which re-schedules jobs every some time, to prevent the problems with reliable Sidekiq.
The description of scheduling algorithm is described in MR. It does not put any significant load on sidekiq, as the most expensive operation is being already done: matching jobs with runners.
Why was this MR needed?
Screenshots (if relevant)
Does this MR meet the acceptance criteria?
- Changelog entry added, if necessary
- Documentation created/updated
- API support added
- Tests added for this feature/bug
- Has been reviewed by UX
- Has been reviewed by Frontend
- Has been reviewed by Backend
- Has been reviewed by Database
- Conform by the merge request performance guides
- Conform by the style guides
- Squashed related commits together
- Internationalization required/considered
End-to-end tests pass (
package-and-qamanual pipeline job)
What are the relevant issue numbers?