Add workhorse image scaler alerting rules
Closes gitlab-org/gitlab-workhorse#297 (closed)
This adds an alert for exceeding the maximum number of image scaler processes we currently allow to run concurrently on any given workhorse node.
I would say that this alert is not critical and should largely serve to inform how this parameter (the number of scalers we can or should run) should be tuned over time. Ideally, we should not see backlogging here, since that would point to:
- scaler being too slow
- scalers not finishing
- node too slow
- too many inbound scaling requests
which all point to some sort of capacity or performance problem
Edited by Matthias Käppler