Skip to content

Add Sidekiq queueing application SLIs

Gregorius Marco requested to merge mg-sidekiq-queueing-sli into master

What does this MR do and why?

Part of gitlab-com/gl-infra/scalability#2305 (closed)

This is similar to when we added Sidekiq execution application SLIs at !116827 (merged)


Add Sidekiq queueing application SLIs

Similar to Sidekiq execution application SLIs, we're adding Sidekiq queueing application SLIs which eventually replaces the sidekiq_jobs_queue_duration_seconds histogram metric.

Changelog: added

How to set up and validate locally

  1. Make sure production.monitoring.sidekiq_exporter.enabled is true

  2. Enable feature flag Feature.enable(:sidekiq_queueing_application_slis)

  3. Restart Sidekiq

    $ gdk restart rails-background-jobs
    ok: down: /Users/gregoriusmarco/Documents/workspace/gdk-10-22/services/rails-background-jobs: 0s
    ok: run: /Users/gregoriusmarco/Documents/workspace/gdk-10-22/services/rails-background-jobs: (pid 8532) 1s, normally down
  4. Check that the metrics are initialized

    $ curl gdk.test:3807/metrics | rg gitlab_sli_sidekiq_queueing_apdex
    ...
  5. Run a job in Rails console Chaos::SleepWorker.perform_async(1)

  6. Check that the counter is incremented properly

    $ curl -s gdk.test:3807/metrics | rg gitlab_sli_sidekiq_queueing_apdex | rg Chaos::SleepWorker
    gitlab_sli_sidekiq_queueing_apdex_success_total{feature_category="not_owned",urgency="low",worker="Chaos::SleepWorker"} 1
    gitlab_sli_sidekiq_queueing_apdex_total{feature_category="not_owned",urgency="low",worker="Chaos::SleepWorker"} 1

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Gregorius Marco

Merge request reports