Sidekiq times out during reconfigure for Omnibus instances upgraded to v18.0.1
Summary
On GitLab instances that have been upgraded multiple times, sudo gitlab-ctl reconfigure fails when Sidekiq doesn't restart within the timeout period.
A workaround for this issue is to first stop GitLab (sudo gitlab-ctl stop), run the reconfigure, and then restart GitLab (sudo gitlab-ctl start). In testing this workaround has been shown to resolve the issue and only needed to be performed once. All following reconfigure commands succeeded without stopping and starting the instance.
Steps to reproduce
Reproduced On Ubuntu 24.04 LTS
- Install GitLab v17.5.5 following the install documentation
- Upgrade to v17.8.7 following the documentation
- Upgrade to v17.11.3 following the documentation
- Upgrade to v18.0.1 following the documentation
- Make a change to the
/etc/gitlab/gitlab.rbthat causes Sidekiq to restart (gitlab_rails['gitlab_default_theme']set to a value between 1 and 11 will work) - Run
sudo gitlab-ctl reconfigure
What is the current bug behavior?
Run sudo gitlab-ctl reconfigure fails because restarting Sidekiq times out.
What is the expected correct behavior?
sudo gitlab-ctl reconfigure should succeed when a valid /etc/gitlab/gitlab.rb is used.
Relevant logs
Relevant logs
Partial output from gitlab-ctl reconfigure
[2025-06-09T18:37:33+00:00] INFO: templatesymlink[Create a gitlab.yml and create a symlink to Rails root] sending restart action to sidekiq_service[sidekiq] (delayed)
Recipe: gitlab::sidekiq
* sidekiq_service[sidekiq] action restart
* service[sidekiq] action nothing (skipped due to action :nothing)
* runit_service[sidekiq] action restart
================================================================================
Error executing action `restart` on resource 'runit_service[sidekiq]'
================================================================================
Mixlib::ShellOut::ShellCommandFailed
------------------------------------
Expected process to exit with [0], but received '1'
---- Begin output of /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/sidekiq ----
STDOUT: timeout: run: /opt/gitlab/service/sidekiq: (pid 87027) 1217s, got TERM
STDERR:
---- End output of /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/sidekiq ----
Ran /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/sidekiq returned 1
Cookbook Trace: (most recent call first)
----------------------------------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/runit/libraries/helpers.rb:136:in `safe_sv_shellout!'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/runit/libraries/helpers.rb:164:in `restart_service'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/runit/libraries/provider_runit_service.rb:368:in `block in '
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/resources/sidekiq_service.rb:67:in `block in class_from_file'
Resource Declaration:
---------------------
# In /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/resources/sidekiq_service.rb
67: runit_service new_resource.name do
68: action :restart
69: end
70: end
Compiled Resource:
------------------
# Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/resources/sidekiq_service.rb:67:in `block in class_from_file'
runit_service("sidekiq") do
action [:restart]
default_guard_interpreter :default
declared_type :runit_service
cookbook_name "gitlab"
recipe_name "sidekiq"
service_name "sidekiq"
owner "root"
group "root"
supports {:restart=>nil, :reload=>nil, :status=>nil}
start_command_name "start"
stop_command_name "stop"
restart_command_name "restart"
status_command_name "status"
end
System Info:
------------
chef_version=18.3.0
platform=ubuntu
platform_version=24.04
ruby=ruby 3.2.5 (2024-07-26 revision 31d0f1a2e7) [x86_64-linux]
program_name=/opt/gitlab/embedded/bin/cinc-client
executable=/opt/gitlab/embedded/bin/cinc-client
================================================================================
Error executing action `restart` on resource 'sidekiq_service[sidekiq]'
================================================================================
Mixlib::ShellOut::ShellCommandFailed
------------------------------------
runit_service[sidekiq] (gitlab::sidekiq line 67) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/sidekiq ----
STDOUT: timeout: run: /opt/gitlab/service/sidekiq: (pid 87027) 1217s, got TERM
STDERR:
---- End output of /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/sidekiq ----
Ran /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/sidekiq returned 1
Cookbook Trace: (most recent call first)
----------------------------------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/runit/libraries/helpers.rb:136:in `safe_sv_shellout!'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/runit/libraries/helpers.rb:164:in `restart_service'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/runit/libraries/provider_runit_service.rb:368:in `block in '
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/resources/sidekiq_service.rb:67:in `block in class_from_file'
Resource Declaration:
---------------------
# In /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/sidekiq.rb
23: sidekiq_service 'sidekiq' do
24: user account_helper.gitlab_user
25: group account_helper.gitlab_group
26: log_directory logging_settings[:log_directory]
27: log_directory_mode logging_settings[:log_directory_mode]
28: log_directory_owner logging_settings[:log_directory_owner]
29: log_directory_group logging_settings[:log_directory_group]
30: log_user logging_settings[:runit_owner]
31: log_group logging_settings[:runit_group]
32: end
33:
Compiled Resource:
------------------
# Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/sidekiq.rb:23:in `from_file'
sidekiq_service("sidekiq") do
action [:enable]
default_guard_interpreter :default
declared_type :sidekiq_service
cookbook_name "gitlab"
recipe_name "sidekiq"
user "git"
group "git"
log_directory "/var/log/gitlab/sidekiq"
log_directory_mode "0700"
log_directory_owner "git"
log_user "root"
logfiles_helper "*sensitive value suppressed*"
template_name "sidekiq"
end
System Info:
------------
chef_version=18.3.0
platform=ubuntu
platform_version=24.04
ruby=ruby 3.2.5 (2024-07-26 revision 31d0f1a2e7) [x86_64-linux]
program_name=/opt/gitlab/embedded/bin/cinc-client
executable=/opt/gitlab/embedded/bin/cinc-client
[2025-06-09T18:38:04+00:00] INFO: templatesymlink[Create a gitlab.yml and create a symlink to Rails root] sending run action to execute[clear the gitlab-rails cache] (delayed)
Recipe: gitlab::gitlab-rails
* execute[clear the gitlab-rails cache] action run[2025-06-09T18:39:13+00:00] INFO: execute[clear the gitlab-rails cache] ran successfully
- execute /opt/gitlab/bin/gitlab-rake cache:clear
[2025-06-09T18:39:13+00:00] INFO: letsencrypt_certificate[reconfigure-testing.jdsandbox.net] sending run action to ruby_block[display_le_message] (delayed)
Recipe: letsencrypt::enable
* ruby_block[display_le_message] action run[2025-06-09T18:39:13+00:00] INFO: ruby_block[display_le_message] called
- execute the ruby block display_le_message
Running handlers:
[2025-06-09T18:39:13+00:00] ERROR: Running exception handlers
There was an error running gitlab-ctl reconfigure:
sidekiq_service[sidekiq] (gitlab::sidekiq line 23) had an error: Mixlib::ShellOut::ShellCommandFailed: runit_service[sidekiq] (gitlab::sidekiq line 67) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/sidekiq ----
STDOUT: timeout: run: /opt/gitlab/service/sidekiq: (pid 87027) 1217s, got TERM
STDERR:
---- End output of /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/sidekiq ----
Ran /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/sidekiq returned 1
Running handlers complete
[2025-06-09T18:39:13+00:00] ERROR: Exception handlers complete
Infra Phase failed. 8 resources updated in 01 minutes 58 seconds
[2025-06-09T18:39:13+00:00] FATAL: Stacktrace dumped to /opt/gitlab/embedded/cookbooks/cache/cinc-stacktrace.out
[2025-06-09T18:39:13+00:00] FATAL: ---------------------------------------------------------------------------------------
[2025-06-09T18:39:13+00:00] FATAL: PLEASE PROVIDE THE CONTENTS OF THE stacktrace.out FILE (above) IF YOU FILE A BUG REPORT
[2025-06-09T18:39:13+00:00] FATAL: ---------------------------------------------------------------------------------------
[2025-06-09T18:39:13+00:00] FATAL: Mixlib::ShellOut::ShellCommandFailed: sidekiq_service[sidekiq] (gitlab::sidekiq line 23) had an error: Mixlib::ShellOut::ShellCommandFailed: runit_service[sidekiq] (gitlab::sidekiq line 67) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/sidekiq ----
STDOUT: timeout: run: /opt/gitlab/service/sidekiq: (pid 87027) 1217s, got TERM
STDERR:
---- End output of /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/sidekiq ----
Ran /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/sidekiq returned 1
cinc-stacktrace.log:
Generated at 2025-06-09 18:39:13 +0000 Mixlib::ShellOut::ShellCommandFailed: sidekiq_service[sidekiq] (gitlab::sidekiq line 23) had an error: Mixlib::ShellOut::ShellCommandFailed: runit_service[sidekiq] (gitlab::sidekiq line 67) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1' ---- Begin output of /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/sidekiq ---- STDOUT: timeout: run: /opt/gitlab/service/sidekiq: (pid 87027) 1217s, got TERM STDERR: ---- End output of /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/sidekiq ---- Ran /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/sidekiq returned 1 /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/mixlib-shellout-3.2.8/lib/mixlib/shellout.rb:300:in `invalid!' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/mixlib-shellout-3.2.8/lib/mixlib/shellout.rb:287:in `error!' :90:in `tap' /opt/gitlab/embedded/cookbooks/cache/cookbooks/runit/libraries/helpers.rb:136:in `safe_sv_shellout!' /opt/gitlab/embedded/cookbooks/cache/cookbooks/runit/libraries/helpers.rb:164:in `restart_service' /opt/gitlab/embedded/cookbooks/cache/cookbooks/runit/libraries/provider_runit_service.rb:368:in `block in ' (eval):2:in `block in action_restart' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/provider.rb:304:in `instance_eval' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/provider.rb:304:in `compile_and_converge_action' (eval):2:in `action_restart' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/provider.rb:245:in `run_action' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/resource.rb:601:in `block in run_action' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/resource.rb:628:in `with_umask' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/resource.rb:600:in `run_action' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/runner.rb:74:in `run_action' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/runner.rb:108:in `block in run_all_actions' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/runner.rb:108:in `each' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/runner.rb:108:in `run_all_actions' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/resource_collection.rb:64:in `insert' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/dsl/declare_resource.rb:267:in `declare_resource' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/dsl/resources.rb:36:in `runit_service' /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/resources/sidekiq_service.rb:67:in `block in class_from_file' (eval):2:in `block in action_restart' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/provider.rb:304:in `instance_eval' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/provider.rb:304:in `compile_and_converge_action' (eval):2:in `action_restart' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/provider.rb:245:in `run_action' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/resource.rb:601:in `block in run_action' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/resource.rb:628:in `with_umask' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/resource.rb:600:in `run_action' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/runner.rb:74:in `run_action' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/runner.rb:168:in `run_delayed_notification' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/runner.rb:155:in `block in run_delayed_notifications' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/runner.rb:154:in `each' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/runner.rb:154:in `run_delayed_notifications' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/runner.rb:144:in `converge' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/client.rb:860:in `block in converge' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/client.rb:855:in `catch' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/client.rb:855:in `converge' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/client.rb:879:in `converge_and_save' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/client.rb:298:in `run' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/application.rb:305:in `run_with_graceful_exit_option' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/application.rb:281:in `block in run_chef_client' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/local_mode.rb:42:in `with_server_connectivity' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/application.rb:264:in `run_chef_client' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/application/base.rb:354:in `run_application' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/application.rb:67:in `run' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-bin-18.3.0/bin/cinc-client:25:in `' /opt/gitlab/embedded/bin/cinc-client:25:in `load' /opt/gitlab/embedded/bin/cinc-client:25:in `' >>>> Caused by Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1' ---- Begin output of /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/sidekiq ---- STDOUT: timeout: run: /opt/gitlab/service/sidekiq: (pid 87027) 1217s, got TERM STDERR: ---- End output of /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/sidekiq ---- Ran /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/sidekiq returned 1 /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/mixlib-shellout-3.2.8/lib/mixlib/shellout.rb:300:in `invalid!' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/mixlib-shellout-3.2.8/lib/mixlib/shellout.rb:287:in `error!' :90:in `tap' /opt/gitlab/embedded/cookbooks/cache/cookbooks/runit/libraries/helpers.rb:136:in `safe_sv_shellout!' /opt/gitlab/embedded/cookbooks/cache/cookbooks/runit/libraries/helpers.rb:164:in `restart_service' /opt/gitlab/embedded/cookbooks/cache/cookbooks/runit/libraries/provider_runit_service.rb:368:in `block in ' (eval):2:in `block in action_restart' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/provider.rb:304:in `instance_eval' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/provider.rb:304:in `compile_and_converge_action' (eval):2:in `action_restart' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/provider.rb:245:in `run_action' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/resource.rb:601:in `block in run_action' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/resource.rb:628:in `with_umask' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/resource.rb:600:in `run_action' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/runner.rb:74:in `run_action' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/runner.rb:108:in `block in run_all_actions' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/runner.rb:108:in `each' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/runner.rb:108:in `run_all_actions' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/resource_collection.rb:64:in `insert' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/dsl/declare_resource.rb:267:in `declare_resource' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/dsl/resources.rb:36:in `runit_service' /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/resources/sidekiq_service.rb:67:in `block in class_from_file' (eval):2:in `block in action_restart' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/provider.rb:304:in `instance_eval' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/provider.rb:304:in `compile_and_converge_action' (eval):2:in `action_restart' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/provider.rb:245:in `run_action' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/resource.rb:601:in `block in run_action' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/resource.rb:628:in `with_umask' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/resource.rb:600:in `run_action' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/runner.rb:74:in `run_action' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/runner.rb:168:in `run_delayed_notification' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/runner.rb:155:in `block in run_delayed_notifications' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/runner.rb:154:in `each' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/runner.rb:154:in `run_delayed_notifications' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/runner.rb:144:in `converge' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/client.rb:860:in `block in converge' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/client.rb:855:in `catch' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/client.rb:855:in `converge' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/client.rb:879:in `converge_and_save' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/client.rb:298:in `run' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/application.rb:305:in `run_with_graceful_exit_option' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/application.rb:281:in `block in run_chef_client' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/local_mode.rb:42:in `with_server_connectivity' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/application.rb:264:in `run_chef_client' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/application/base.rb:354:in `run_application' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-18.3.0/lib/chef/application.rb:67:in `run' /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/chef-bin-18.3.0/bin/cinc-client:25:in `' /opt/gitlab/embedded/bin/cinc-client:25:in `load' /opt/gitlab/embedded/bin/cinc-client:25:in `'
Output from gitlab-ctl tail sidekiq while the error occurs:
{"severity":"INFO","time":"2025-06-09T18:37:05.800Z","message":"No existing merge request to be cleaned up."}
{"severity":"INFO","time":"2025-06-09T18:37:05.804Z","message":"No existing merge request to be cleaned up."}
{"severity":"INFO","time":"2025-06-09T18:37:05.813Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c42b3ce7efee6079e57ca758","created_at":"2025-06-09T18:37:04.700Z","trace_propagation_headers":{"sentry-trace":"fa1e8cb1e6d347618995e92b68f107f1-3eafd036faaa40c8","baggage":"sentry-trace_id=fa1e8cb1e6d347618995e92b68f107f1,sentry-release=3426be1b938"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"48abfcbf689c200f4ebc7115008f9d0f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2025-06-09T18:37:04.707Z","job_size_bytes":2,"pid":87039,"sidekiq_tid":"h9cb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c42b3ce7efee6079e57ca758: done: 0.255063 sec","job_status":"done","queue_duration_s":0.850383,"scheduling_latency_s":0.850414,"redis_calls":6,"redis_duration_s":0.033854999999999996,"redis_read_bytes":6,"redis_write_bytes":588,"redis_queues_metadata_calls":1,"redis_queues_metadata_duration_s":0.000247,"redis_queues_metadata_read_bytes":1,"redis_queues_metadata_write_bytes":49,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.033608,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":539,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.045,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.045,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.045,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.045,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014688,"mem_objects":3256,"mem_bytes":56944,"mem_mallocs":660,"mem_total_bytes":187184,"worker_id":"sidekiq_0","rate_limiting_gates":[],"duration_s":0.255063,"completed_at":"2025-06-09T18:37:05.812Z","load_balancing_strategy":"primary","db_duration_s":0.101615,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}
{"severity":"INFO","time":"2025-06-09T18:37:05.823Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c5b0a1da56d4c0c8456f962b","created_at":"2025-06-09T18:37:04.700Z","trace_propagation_headers":{"sentry-trace":"fa1e8cb1e6d347618995e92b68f107f1-3eafd036faaa40c8","baggage":"sentry-trace_id=fa1e8cb1e6d347618995e92b68f107f1,sentry-release=3426be1b938"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"48abfcbf689c200f4ebc7115008f9d0f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2025-06-09T18:37:04.707Z","job_size_bytes":2,"pid":87039,"sidekiq_tid":"h967","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c5b0a1da56d4c0c8456f962b: done: 0.137642 sec","job_status":"done","queue_duration_s":0.978161,"scheduling_latency_s":0.978194,"redis_calls":6,"redis_duration_s":0.001912,"redis_read_bytes":6,"redis_write_bytes":588,"redis_queues_metadata_calls":1,"redis_queues_metadata_duration_s":0.000326,"redis_queues_metadata_read_bytes":1,"redis_queues_metadata_write_bytes":49,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.001586,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":539,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.015,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.015,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.031,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014771,"mem_objects":3256,"mem_bytes":56944,"mem_mallocs":660,"mem_total_bytes":187184,"worker_id":"sidekiq_0","rate_limiting_gates":[],"duration_s":0.137642,"completed_at":"2025-06-09T18:37:05.823Z","load_balancing_strategy":"primary","db_duration_s":0.054349,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}
{"severity":"INFO","time":"2025-06-09T18:37:05.850Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b79a8901efc3746fd8bed9e3","created_at":"2025-06-09T18:37:04.700Z","trace_propagation_headers":{"sentry-trace":"fa1e8cb1e6d347618995e92b68f107f1-3eafd036faaa40c8","baggage":"sentry-trace_id=fa1e8cb1e6d347618995e92b68f107f1,sentry-release=3426be1b938"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"48abfcbf689c200f4ebc7115008f9d0f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2025-06-09T18:37:04.707Z","job_size_bytes":2,"pid":87039,"sidekiq_tid":"h9br","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b79a8901efc3746fd8bed9e3: done: 0.255675 sec","job_status":"done","queue_duration_s":0.887523,"scheduling_latency_s":0.887555,"redis_calls":6,"redis_duration_s":0.064986,"redis_read_bytes":6,"redis_write_bytes":588,"redis_queues_metadata_calls":1,"redis_queues_metadata_duration_s":0.016293,"redis_queues_metadata_read_bytes":1,"redis_queues_metadata_write_bytes":49,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.048693,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":539,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.056,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.068,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.068,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.056,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015294,"mem_objects":3261,"mem_bytes":56784,"mem_mallocs":658,"mem_total_bytes":187224,"worker_id":"sidekiq_0","rate_limiting_gates":[],"duration_s":0.255675,"completed_at":"2025-06-09T18:37:05.850Z","load_balancing_strategy":"primary","db_duration_s":0.132604,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}
{"severity":"INFO","time":"2025-06-09T18:37:05.870Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ba800199aa94837636eff8e6","created_at":"2025-06-09T18:37:04.700Z","trace_propagation_headers":{"sentry-trace":"fa1e8cb1e6d347618995e92b68f107f1-3eafd036faaa40c8","baggage":"sentry-trace_id=fa1e8cb1e6d347618995e92b68f107f1,sentry-release=3426be1b938"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"48abfcbf689c200f4ebc7115008f9d0f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2025-06-09T18:37:04.707Z","job_size_bytes":2,"pid":87039,"sidekiq_tid":"h9a3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ba800199aa94837636eff8e6: done: 0.182611 sec","job_status":"done","queue_duration_s":0.979999,"scheduling_latency_s":0.980026,"redis_calls":6,"redis_duration_s":0.080013,"redis_read_bytes":6,"redis_write_bytes":588,"redis_queues_metadata_calls":1,"redis_queues_metadata_duration_s":0.002623,"redis_queues_metadata_read_bytes":1,"redis_queues_metadata_write_bytes":49,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.07739,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":539,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.045,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.015,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.015,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.045,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014843,"mem_objects":3256,"mem_bytes":56784,"mem_mallocs":658,"mem_total_bytes":187024,"worker_id":"sidekiq_0","rate_limiting_gates":[],"duration_s":0.182611,"completed_at":"2025-06-09T18:37:05.870Z","load_balancing_strategy":"primary","db_duration_s":0.067752,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}
{"severity":"INFO","time":"2025-06-09T18:37:06.273Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"ebc17c71d749eb7c06858777","created_at":"2025-06-09T18:37:03.180Z","trace_propagation_headers":{"sentry-trace":"fa1e8cb1e6d347618995e92b68f107f1-33500a4b8e2d4b06","baggage":"sentry-trace_id=fa1e8cb1e6d347618995e92b68f107f1,sentry-environment=,sentry-release=3426be1b938"},"meta.caller_id":"Cronjob","correlation_id":"bf2406bf2ebdc3acf474f37e46caff30","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2025-06-09T18:37:03.183Z","job_size_bytes":2,"pid":87039,"sidekiq_tid":"h97b","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-ebc17c71d749eb7c06858777: done: 3.087326 sec","job_status":"done","queue_duration_s":0.003165,"scheduling_latency_s":0.003189,"redis_calls":871,"redis_duration_s":2.71706,"redis_read_bytes":985,"redis_write_bytes":77469,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.003063,"redis_feature_flag_read_bytes":115,"redis_feature_flag_write_bytes":83,"redis_queues_metadata_calls":2,"redis_queues_metadata_duration_s":0.001596,"redis_queues_metadata_read_bytes":2,"redis_queues_metadata_write_bytes":172,"redis_shared_state_calls":868,"redis_shared_state_duration_s":2.712401,"redis_shared_state_read_bytes":868,"redis_shared_state_write_bytes":77214,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.437652,"mem_objects":91997,"mem_bytes":1381680,"mem_mallocs":20795,"mem_total_bytes":5061560,"worker_id":"sidekiq_0","rate_limiting_gates":[],"duration_s":3.087326,"completed_at":"2025-06-09T18:37:06.273Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}
{"severity":"INFO","time":"2025-06-09T18:37:33.311Z","message":"Shutting down"}
{"severity":"INFO","time":"2025-06-09T18:37:33.311Z","message":"Scheduler exiting..."}
{"severity":"INFO","time":"2025-06-09T18:37:33.312Z","message":"Terminating quiet threads for default capsule"}
{"severity":"INFO","time":"2025-06-09T18:37:33.312Z","message":"Scheduler exiting..."}
{"severity":"INFO","time":"2025-06-09T18:37:33.813Z","message":"Pausing to allow jobs to finish..."}
{"severity":"INFO","time":"2025-06-09T18:37:36.815Z","message":"Bye!"}
[2025-06-09 18:37:36] INFO going to shutdown ...
[2025-06-09 18:37:36] INFO WEBrick::HTTPServer#start done.
{"severity":"INFO","time":"2025-06-09T18:37:36.818Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":87039,"worker_id":"sidekiq_0","memwd_rss_bytes":1276575744,"retry":0}
{"severity":"INFO","time":"2025-06-09T18:38:14.582Z","message":"Starting cluster with 1 processes"}
{"severity":"INFO","time":"2025-06-09T18:38:14.627Z","message":"Starting metrics server on port 8082"}
2025-06-09T18:39:22.913Z pid=88960 tid=2mp0 INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>88960, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>401096704}
{"severity":"INFO","time":"2025-06-09T18:39:29.047Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}
{"severity":"INFO","time":"2025-06-09T18:39:35.211Z","message":"GitLab reliable fetch activated!"}
{"severity":"INFO","time":"2025-06-09T18:39:35.212Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e25, :pool_name=\u003e\"internal\", :path=\u003e\"/var/opt/gitlab/redis/redis.socket\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}
{"severity":"INFO","time":"2025-06-09T18:39:35.362Z","message":"Cron Jobs - added job with name: stuck_ci_jobs_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.366Z","message":"Cron Jobs - added job with name: pipeline_schedule_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.370Z","message":"Cron Jobs - added job with name: expire_build_artifacts_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.373Z","message":"Cron Jobs - added job with name: update_locked_unknown_artifacts_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.378Z","message":"Cron Jobs - added job with name: ci_partitioning_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.382Z","message":"Cron Jobs - added job with name: ci_pipelines_expire_artifacts_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.385Z","message":"Cron Jobs - added job with name: ci_schedule_delete_objects_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.389Z","message":"Cron Jobs - added job with name: environments_auto_stop_cron_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.394Z","message":"Cron Jobs - added job with name: environments_auto_delete_cron_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.399Z","message":"Cron Jobs - added job with name: repository_check_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.404Z","message":"Cron Jobs - added job with name: admin_email_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.409Z","message":"Cron Jobs - added job with name: personal_access_tokens_expiring_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.414Z","message":"Cron Jobs - added job with name: personal_access_tokens_expired_notification_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.418Z","message":"Cron Jobs - added job with name: resource_access_tokens_inactive_tokens_deletion_cron_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.423Z","message":"Cron Jobs - added job with name: repository_archive_cache_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.428Z","message":"Cron Jobs - added job with name: import_export_project_cleanup_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.432Z","message":"Cron Jobs - added job with name: gitlab_import_import_file_cleanup_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.436Z","message":"Cron Jobs - added job with name: ci_archive_traces_cron_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.440Z","message":"Cron Jobs - added job with name: members_expiring_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.444Z","message":"Cron Jobs - added job with name: remove_expired_members_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.448Z","message":"Cron Jobs - added job with name: remove_expired_group_links_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.451Z","message":"Cron Jobs - added job with name: remove_unaccepted_member_invites_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.456Z","message":"Cron Jobs - added job with name: prune_old_events_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.461Z","message":"Cron Jobs - added job with name: gitlab_export_prune_project_export_jobs_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.465Z","message":"Cron Jobs - added job with name: trending_projects_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.470Z","message":"Cron Jobs - added job with name: remove_unreferenced_lfs_objects_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.474Z","message":"Cron Jobs - added job with name: bulk_imports_stale_import_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.478Z","message":"Cron Jobs - added job with name: import_stuck_project_import_jobs"}
{"severity":"INFO","time":"2025-06-09T18:39:35.483Z","message":"Cron Jobs - added job with name: jira_import_stuck_jira_import_jobs"}
{"severity":"INFO","time":"2025-06-09T18:39:35.487Z","message":"Cron Jobs - added job with name: stuck_export_jobs_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.491Z","message":"Cron Jobs - added job with name: gitlab_service_ping_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.496Z","message":"Cron Jobs - added job with name: stuck_merge_jobs_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.499Z","message":"Cron Jobs - added job with name: pages_domain_verification_cron_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.504Z","message":"Cron Jobs - added job with name: pages_domain_removal_cron_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.508Z","message":"Cron Jobs - added job with name: pages_domain_ssl_renewal_cron_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.512Z","message":"Cron Jobs - added job with name: issue_due_scheduler_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.517Z","message":"Cron Jobs - added job with name: schedule_migrate_external_diffs_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.521Z","message":"Cron Jobs - added job with name: namespaces_prune_aggregation_schedules_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.527Z","message":"Cron Jobs - added job with name: container_expiration_policy_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.531Z","message":"Cron Jobs - added job with name: cleanup_container_registry_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.536Z","message":"Cron Jobs - added job with name: image_ttl_group_policy_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.542Z","message":"Cron Jobs - added job with name: cleanup_dependency_proxy_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.546Z","message":"Cron Jobs - added job with name: cleanup_package_registry_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.550Z","message":"Cron Jobs - added job with name: x509_issuer_crl_check_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.554Z","message":"Cron Jobs - added job with name: users_create_statistics_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.559Z","message":"Cron Jobs - added job with name: authorized_project_update_periodic_recalculate_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.563Z","message":"Cron Jobs - added job with name: update_container_registry_info_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.566Z","message":"Cron Jobs - added job with name: postgres_dynamic_partitions_manager"}
{"severity":"INFO","time":"2025-06-09T18:39:35.570Z","message":"Cron Jobs - added job with name: postgres_dynamic_partitions_dropper"}
{"severity":"INFO","time":"2025-06-09T18:39:35.574Z","message":"Cron Jobs - added job with name: analytics_usage_trends_count_job_trigger_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.578Z","message":"Cron Jobs - added job with name: member_invitation_reminder_emails_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.581Z","message":"Cron Jobs - added job with name: schedule_merge_request_cleanup_refs_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.584Z","message":"Cron Jobs - added job with name: manage_evidence_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.589Z","message":"Cron Jobs - added job with name: publish_release_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.592Z","message":"Cron Jobs - added job with name: user_status_cleanup_batch_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.597Z","message":"Cron Jobs - added job with name: ssh_keys_expired_notification_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.602Z","message":"Cron Jobs - added job with name: ssh_keys_expiring_soon_notification_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.606Z","message":"Cron Jobs - added job with name: users_deactivate_dormant_users_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.611Z","message":"Cron Jobs - added job with name: ci_delete_unit_tests_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.615Z","message":"Cron Jobs - added job with name: batched_background_migrations_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.619Z","message":"Cron Jobs - added job with name: batched_background_migration_worker_ci_database"}
{"severity":"INFO","time":"2025-06-09T18:39:35.623Z","message":"Cron Jobs - added job with name: issues_reschedule_stuck_issue_rebalances"}
{"severity":"INFO","time":"2025-06-09T18:39:35.628Z","message":"Cron Jobs - added job with name: projects_schedule_refresh_build_artifacts_size_statistics_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.632Z","message":"Cron Jobs - added job with name: inactive_projects_deletion_cron_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.636Z","message":"Cron Jobs - added job with name: loose_foreign_keys_cleanup_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.640Z","message":"Cron Jobs - added job with name: batched_git_ref_updates_cleanup_scheduler_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.644Z","message":"Cron Jobs - added job with name: ci_runner_versions_reconciliation_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.649Z","message":"Cron Jobs - added job with name: users_migrate_records_to_ghost_user_in_batches_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.653Z","message":"Cron Jobs - added job with name: ci_runners_stale_machines_cleanup_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.658Z","message":"Cron Jobs - added job with name: packages_cleanup_delete_orphaned_dependencies_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.662Z","message":"Cron Jobs - added job with name: cleanup_dangling_debian_package_files_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.666Z","message":"Cron Jobs - added job with name: object_storage_delete_stale_direct_uploads_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.670Z","message":"Cron Jobs - added job with name: service_desk_custom_email_verification_cleanup"}
{"severity":"INFO","time":"2025-06-09T18:39:35.674Z","message":"Cron Jobs - added job with name: deactivated_pages_deployments_delete_cron_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.679Z","message":"Cron Jobs - added job with name: ci_schedule_unlock_pipelines_in_queue_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.683Z","message":"Cron Jobs - added job with name: ci_catalog_resources_process_sync_events_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.687Z","message":"Cron Jobs - added job with name: namespaces_process_outdated_namespace_descendants_cron_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.691Z","message":"Cron Jobs - added job with name: performance_bar_stats"}
{"severity":"INFO","time":"2025-06-09T18:39:35.695Z","message":"Cron Jobs - added job with name: ci_catalog_resources_aggregate_last30_day_usage_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.699Z","message":"Cron Jobs - added job with name: ci_catalog_resources_cleanup_last_usages_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.703Z","message":"Cron Jobs - added job with name: ci_click_house_finished_pipelines_sync_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.708Z","message":"Cron Jobs - added job with name: deactivate_expired_deployments_cron_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.712Z","message":"Cron Jobs - added job with name: database_monitor_locked_tables_cron_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.716Z","message":"Cron Jobs - added job with name: merge_requests_process_scheduled_merge"}
{"severity":"INFO","time":"2025-06-09T18:39:35.720Z","message":"Cron Jobs - added job with name: ci_schedule_old_pipelines_removal_cron_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.724Z","message":"Cron Jobs - added job with name: version_version_check_cron"}
{"severity":"INFO","time":"2025-06-09T18:39:35.729Z","message":"Cron Jobs - added job with name: import_placeholder_user_cleanup_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.732Z","message":"Cron Jobs - added job with name: analytics_devops_adoption_create_all_snapshots_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.737Z","message":"Cron Jobs - added job with name: analytics_cycle_analytics_incremental_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.741Z","message":"Cron Jobs - added job with name: analytics_cycle_analytics_consistency_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.744Z","message":"Cron Jobs - added job with name: analytics_cycle_analytics_reaggregation_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.749Z","message":"Cron Jobs - added job with name: analytics_cycle_analytics_stage_aggregation_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.754Z","message":"Cron Jobs - added job with name: analytics_value_stream_dashboard_count_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.758Z","message":"Cron Jobs - added job with name: active_user_count_threshold_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.763Z","message":"Cron Jobs - added job with name: adjourned_group_deletion_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.768Z","message":"Cron Jobs - added job with name: adjourned_projects_deletion_cron_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.772Z","message":"Cron Jobs - added job with name: geo_verification_cron_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.776Z","message":"Cron Jobs - added job with name: geo_sync_timeout_cron_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.781Z","message":"Cron Jobs - added job with name: geo_secondary_usage_data_cron_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.785Z","message":"Cron Jobs - added job with name: geo_registry_sync_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.789Z","message":"Cron Jobs - added job with name: geo_repository_registry_sync_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.794Z","message":"Cron Jobs - added job with name: geo_metrics_update_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.798Z","message":"Cron Jobs - added job with name: geo_prune_event_log_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.803Z","message":"Cron Jobs - added job with name: geo_secondary_registry_consistency_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.807Z","message":"Cron Jobs - added job with name: historical_data_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.811Z","message":"Cron Jobs - added job with name: incident_sla_exceeded_check_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.815Z","message":"Cron Jobs - added job with name: incident_management_persist_oncall_rotation_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.821Z","message":"Cron Jobs - added job with name: incident_management_schedule_escalation_check_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.826Z","message":"Cron Jobs - added job with name: import_software_licenses_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.830Z","message":"Cron Jobs - added job with name: ldap_group_sync_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.834Z","message":"Cron Jobs - added job with name: ldap_sync_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.839Z","message":"Cron Jobs - added job with name: ldap_admin_sync_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.843Z","message":"Cron Jobs - added job with name: queue_refresh_of_broken_adherence_groups_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.855Z","message":"Cron Jobs - added job with name: elastic_index_bulk_cron_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.861Z","message":"Cron Jobs - added job with name: elastic_index_embedding_bulk_cron_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.866Z","message":"Cron Jobs - added job with name: elastic_index_initial_bulk_cron_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.870Z","message":"Cron Jobs - added job with name: elastic_cluster_reindexing_cron_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.875Z","message":"Cron Jobs - added job with name: elastic_remove_expired_namespace_subscriptions_from_index_cron_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.880Z","message":"Cron Jobs - added job with name: elastic_migration_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.884Z","message":"Cron Jobs - added job with name: search_zoekt_scheduling_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.889Z","message":"Cron Jobs - added job with name: search_zoekt_rollout_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.893Z","message":"Cron Jobs - added job with name: search_elastic_metrics_update_cron_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.899Z","message":"Cron Jobs - added job with name: search_elastic_migration_cleanup_cron_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.903Z","message":"Cron Jobs - added job with name: search_zoekt_metrics_update_cron_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.908Z","message":"Cron Jobs - added job with name: pause_control_resume_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.912Z","message":"Cron Jobs - added job with name: concurrency_limit_resume_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.918Z","message":"Cron Jobs - added job with name: sync_seat_link_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.923Z","message":"Cron Jobs - added job with name: sync_service_token_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.927Z","message":"Cron Jobs - added job with name: iterations_update_status_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.932Z","message":"Cron Jobs - added job with name: iterations_generator_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.936Z","message":"Cron Jobs - added job with name: vulnerability_archival_schedule_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.941Z","message":"Cron Jobs - added job with name: vulnerability_statistics_schedule_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.945Z","message":"Cron Jobs - added job with name: vulnerability_namespace_statistics_schedule_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.950Z","message":"Cron Jobs - added job with name: vulnerability_historical_statistics_deletion_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.955Z","message":"Cron Jobs - added job with name: vulnerability_orphaned_remediations_cleanup_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.959Z","message":"Cron Jobs - added job with name: security_create_orchestration_policy_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.963Z","message":"Cron Jobs - added job with name: security_orchestration_policy_rule_schedule_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.968Z","message":"Cron Jobs - added job with name: security_pipeline_execution_policies_schedule_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.973Z","message":"Cron Jobs - added job with name: security_scans_purge_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.978Z","message":"Cron Jobs - added job with name: app_sec_dast_profile_schedule_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.982Z","message":"Cron Jobs - added job with name: ci_namespace_mirrors_consistency_check_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.987Z","message":"Cron Jobs - added job with name: ci_project_mirrors_consistency_check_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.992Z","message":"Cron Jobs - added job with name: arkose_blocked_users_report_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:35.997Z","message":"Cron Jobs - added job with name: ci_runners_stale_group_runners_prune_worker_cron"}
{"severity":"INFO","time":"2025-06-09T18:39:36.002Z","message":"Cron Jobs - added job with name: licenses_reset_submit_license_usage_data_banner"}
{"severity":"INFO","time":"2025-06-09T18:39:36.007Z","message":"Cron Jobs - added job with name: package_metadata_licenses_sync_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:36.011Z","message":"Cron Jobs - added job with name: compliance_violations_consistency_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:36.017Z","message":"Cron Jobs - added job with name: framework_evaluation_scheduler_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:36.025Z","message":"Cron Jobs - added job with name: users_delete_unconfirmed_users_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:36.034Z","message":"Cron Jobs - added job with name: users_unconfirmed_secondary_emails_deletion_cron_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:36.043Z","message":"Cron Jobs - added job with name: package_metadata_advisories_sync_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:36.051Z","message":"Cron Jobs - added job with name: okr_checkin_reminder_emails"}
{"severity":"INFO","time":"2025-06-09T18:39:36.056Z","message":"Cron Jobs - added job with name: timeout_pending_status_check_responses_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:36.060Z","message":"Cron Jobs - added job with name: click_house_ci_finished_builds_sync_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:36.064Z","message":"Cron Jobs - added job with name: click_house_events_sync_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:36.068Z","message":"Cron Jobs - added job with name: click_house_user_add_on_assignments_sync_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:36.072Z","message":"Cron Jobs - added job with name: click_house_event_authors_consistency_cron_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:36.076Z","message":"Cron Jobs - added job with name: click_house_event_namespace_paths_consistency_cron_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:36.080Z","message":"Cron Jobs - added job with name: click_house_rebuild_materialized_view_cron_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:36.084Z","message":"Cron Jobs - added job with name: click_house_dump_all_write_buffers_cron_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:36.088Z","message":"Cron Jobs - added job with name: gitlab_subscriptions_add_on_purchases_schedule_bulk_refresh_user_assignments_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:36.093Z","message":"Cron Jobs - added job with name: gitlab_subscriptions_add_on_purchases_cleanup_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:36.097Z","message":"Cron Jobs - added job with name: gitlab_subscriptions_offline_cloud_license_provision_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:36.100Z","message":"Cron Jobs - added job with name: observability_alert_query_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:36.104Z","message":"Cron Jobs - added job with name: report_security_policies_metrics_worker.rb"}
{"severity":"INFO","time":"2025-06-09T18:39:36.108Z","message":"Cron Jobs - added job with name: usage_events_dump_write_buffer_cron_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:36.111Z","message":"Cron Jobs - added job with name: package_metadata_cve_enrichment_sync_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:36.116Z","message":"Cron Jobs - added job with name: members_schedule_prune_deletions_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:36.122Z","message":"Cron Jobs - added job with name: ai_conversation_cleanup_cron_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:36.126Z","message":"Cron Jobs - added job with name: ai_active_context_bulk_process_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:36.130Z","message":"Cron Jobs - added job with name: ai_active_context_migration_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:36.135Z","message":"Cron Jobs - added job with name: namespaces_enable_descendants_cache_cron_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:36.139Z","message":"Cron Jobs - added job with name: delete_expired_dependency_exports_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:36.143Z","message":"Cron Jobs - added job with name: analytics_dump_ai_user_metrics_database_write_buffer_cron_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:36.148Z","message":"Cron Jobs - added job with name: delete_expired_vulnerability_exports_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:36.152Z","message":"Cron Jobs - added job with name: ai_duo_workflows_fail_stuck_workflows_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:36.158Z","message":"Cron Jobs - deleted job with name: update_all_mirrors_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:36.164Z","message":"Cron Jobs - added job with name: update_all_mirrors_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:36.170Z","message":"Cron Jobs - deleted job with name: geo_sidekiq_cron_config_worker"}
{"severity":"INFO","time":"2025-06-09T18:39:36.174Z","message":"Cron Jobs - added job with name: geo_sidekiq_cron_config_worker"}
Details of package version
Provide the package version installation details
external_url ''
letsencrypt['contact_emails'] = ['jdiaz@gitlab.com']
gitlab_rails['env'] = {
"GITLAB_LICENSE_MODE" => "test",
"CUSTOMER_PORTAL_URL" => "https://customers.staging.gitlab.com",
"AI_GATEWAY_URL" => "https://cloud.staging.gitlab.com/ai"
}
gitlab_rails['gitlab_default_theme'] = 1