Skip to content

Remove initialization of unneeded metrics for Sidekiq processes

What does this MR do?

Avoid initialization of http_request_duration_seconds histogram for Sidekiq servers.

  • The sidekiq processes do not need to report the http_request_duration_seconds prometheus metric that is being initialized in RequestsRackMiddleware.initialize_http_request_duration_seconds since that metric is a webserver-centric metric

  • The collection of http_request_duration_seconds causes 1200+ lines of zero or almost zero output for the sidekiq-exporter process

  • The initialization of http_request_duration_seconds causes undue accumulation of unneeded metrics in the multiprocess shared files

  • Now that the sidekiq-cluster process will be tracking metrics for all created sidekiq processes there will be more disk space utilization, due to the storage of these unneeded metrics for each of the sidekiq pids.

Thanos query that I think illustrates the current state:

count_values of http_request_duration_seconds buckets

Screenshots

Sidekiq /metrics endpoint before removing this initialization (1665 lines) (cleaned metrics directory)

before-metrics-2.txt

Sidekiq /metrics endpoint after removing this initialization (448 lines) (cleaned metrics directory)

after-metrics.txt

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

If some user was expecting to get these unneeded metrics from Sidekiq, this might be surprising. This webserver metric is not a valid measurement for sidekiq processes, though.

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • [-] Label as security and @ mention @gitlab-com/gl-security/appsec
  • [-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • [-] Security reports checked/validated by a reviewer from the AppSec team
Edited by 🤖 GitLab Bot 🤖

Merge request reports