Draft: Cleanup and replace sidekiq rules
This MR has been split up in 5 MRs:
- Remove useless Sidekiq related dashboards (!6089 - merged)
- Add queue as significant label in Sidekiq SLIs (!6090 - merged)
- Clean up per-worker Sidekiq recording rules (!6091 - merged)
- Redirect Sidekiq job latencies to Kibana links (!6092 - merged)
- Remove usage of Sidekiq histogram metrics in me... (!6096 - merged)
This MR aims to clean up:
- Sidekiq per-worker recording rules in
metrics-catalog/services/lib/sidekiq-per-worker-recording-rules.libsonnet
, i.e. those withgitlab_background_jobs:*
- Replace the intermediate rules in https://gitlab.com/gitlab-com/runbooks/-/blob/495f7edd799917bcfe7db4f85a29039036d1acf4/metrics-catalog/services/sidekiq.jsonnet#L65-66, i.e. references to
sli_aggregations:sidekiq_jobs_completion_seconds_bucket
andsli_aggregations:sidekiq_jobs_queue_duration_seconds_bucket
. Panels with queueing & execution latency will be directed to Kibana visualization instead (which is much more accurate but only having 7 days retention period).
Part of the cleanup in rollout plan gitlab-com/gl-infra/scalability#2415 (closed)
More context: https://gitlab.com/gitlab-com/runbooks/-/blob/master/docs/sidekiq/sidekiq-slis.md
To test the dashboards:
❯ ./test-dashboard.sh sidekiq/worker-detail.dashboard.jsonnet
Installed https://dashboards.gitlab.net/dashboard/snapshot/eQDhUmfs7WvFX7iZTVpmfwdEElzTx1QP - sidekiq: Worker Detail
❯ ./test-dashboard.sh sidekiq/queue-detail.dashboard.jsonnet
Installed https://dashboards.gitlab.net/dashboard/snapshot/X2VPGBIZzmlEIIxS7hWb53coTDPpzfbd - sidekiq: Queue Detail
❯ ./test-dashboard.sh sidekiq/shard-detail.dashboard.jsonnet
Installed https://dashboards.gitlab.net/dashboard/snapshot/kpAJ7Nes8DPXlT4wVU7YupCgKv1SryhV - sidekiq: Shard Detail
Edited by Gregorius Marco