Skip to content

`gitlab-ctl hup sidekiq/sidekiq-ctl` does not perform graceful shutdown

Today we use gitlab-ctl hup sidekiq/sidekiq-cluster as stated in our documentation to perform graceful shutdown of a the sidekiq process. However, this is not implemented in a way as described.

The HUP signal is not really handled by Sidekiq. It really interrupts the process without:

  1. completing any jobs,
  2. interrupting any jobs,
  3. pushing jobs to the queue,
  4. HUP effectively makes us to loose all jobs currently in-flight.

The proper way for sidekiq to operate is to use the:

  1. SIGINT: Ctrl-C from terminal,
  2. SIGTERM,

That results in the following message:

{"severity":"INFO","time":"2019-12-16T12:36:50.904Z","message":"Shutting down"}
{"severity":"INFO","time":"2019-12-16T12:36:50.904Z","message":"Scheduler exiting..."}
{"severity":"INFO","time":"2019-12-16T12:36:50.904Z","message":"Terminating quiet workers"}
{"severity":"INFO","time":"2019-12-16T12:36:50.904Z","message":"Scheduler exiting..."}
{"severity":"INFO","time":"2019-12-16T12:36:51.405Z","message":"Pausing to allow workers to finish..."}
{"severity":"INFO","time":"2019-12-16T12:36:53.407Z","message":"Bye!"}

The default timeout gives us 8 seconds for jobs to finish, otherwise they are interrupted and pushed back into the queue.

Edited by Kamil Trzciński