Skip to content

Sidekiq status is misleading when GitLab is running in a HA environment

Summary

Per https://gitlab.com/gitlab-com/infrastructure/issues/4161

When running GitLab in a HA mode, sidekiq status is not reported correctly. Instance admins are told sidekiq is not running, which is extremely disconcerting.

Steps to reproduce

Set up a GitLab instance using... sidekiq-cluster? Sidekiq running on a separate server?

What is the current bug behavior?

A scary message: https://gitlab.com/gitlab-com/infrastructure/uploads/3aae150315079bac82ef875cd30f0948/Screenshot_from_2018-05-03_15-31-02.png -> Broken link

image

What is the expected correct behavior?

We should at least see "Detailed sidekiq information is not available because you are running in HA mode", rather than the scary text.

Output of checks

This bugs happens on [staging,gstg].gitlab.com

Possible fixes

Ideally, we'd push the status into redis from the sidekiq nodes and display the complete sidekiq cluster state whether sidekiq is running locally or remotely. I have the horrible feeling that we're using ps to get the state at present.

Edited by Andrew Winata