Sidekiq lease errors concurrency_limit_sampler after gitlab upgrade from 17.11.6 to 18.2.5

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

Summary

After upgrading our GitLab EE instance from 17.11.6 to 18.2.5, we started seeing repeated Cannot obtain an exclusive lease errors in the logs related to Gitlab::Metrics::Samplers::ConcurrencyLimitSampler. This did not occur prior to the upgrade.

Observed Logs

{"severity":"ERROR","time":"2025-09-09T08:46:33.927Z","message":"Cannot obtain an exclusive lease. There must be another instance already in execution.","lease_key":"gitlab/metrics/samplers/concurrency_limit_sampler","class_name":"Gitlab::Metrics::Samplers::ConcurrencyLimitSampler","lease_timeout":300}
{"severity":"ERROR","time":"2025-09-09T08:46:35.474Z","message":"Cannot obtain an exclusive lease. There must be another instance already in execution.","lease_key":"gitlab/metrics/samplers/concurrency_limit_sampler","class_name":"Gitlab::Metrics::Samplers::ConcurrencyLimitSampler","lease_timeout":300}
{"severity":"ERROR","time":"2025-09-09T08:46:38.834Z","message":"Cannot obtain an exclusive lease. There must be another instance already in execution.","lease_key":"gitlab/metrics/samplers/concurrency_limit_sampler","class_name":"Gitlab::Metrics::Samplers::ConcurrencyLimitSampler","lease_timeout":300}
{"severity":"ERROR","time":"2025-09-09T08:46:39.531Z","message":"Cannot obtain an exclusive lease. There must be another instance already in execution.","lease_key":"gitlab/metrics/samplers/concurrency_limit_sampler","class_name":"Gitlab::Metrics::Samplers::ConcurrencyLimitSampler","lease_timeout":300}
{"severity":"ERROR","time":"2025-09-09T08:46:40.135Z","message":"Cannot obtain an exclusive lease. There must be another instance already in execution.","lease_key":"gitlab/metrics/samplers/concurrency_limit_sampler","class_name":"Gitlab::Metrics::Samplers::ConcurrencyLimitSampler","lease_timeout":300}

Environment

  • Edition: GitLab EE
  • Deployment: Docker
  • Previous version: 17.11.6
  • Current version: 18.2.5

Sidekiq Configuration

################################################################################
## GitLab Sidekiq
################################################################################

sidekiq['enable'] = true
sidekiq['log_directory'] = "/var/log/gitlab/sidekiq"
sidekiq['log_format'] = "json"
sidekiq['shutdown_timeout'] = 4
sidekiq['interval'] = 5
# sidekiq['concurrency'] = nil
# sidekiq['max_concurrency'] = 20
# sidekiq['min_concurrency'] = nil

# sidekiq['routing_rules'] = []

# We configured 8 sidekiq processes, each processing all queues
sidekiq['queue_groups'] = ['*'] * 8

Request

Can you provide guidance or a fix to prevent the repeated Cannot obtain an exclusive lease errors from flooding the logs?

Edited by 🤖 GitLab Bot 🤖