Skip to content

Add feature-flagged "channel per key" implementation for CI long polling

Jacob Vosmaer requested to merge jv-workhorse-pubsub into master

gitlab-com/gl-infra/scalability#1900 (closed)

In gitlab-com/gl-infra&784 (closed) we discovered that the current implementation of CI runner long polling is very wasteful with Redis resources. We deliver 75K messages per second from Rails to Workhorse and 99.9% of those get dropped by Workhorse.

This MR adds a feature-flagged alternative implementation where we use one Redis Pub/Sub channel per key.

Merge request reports