WIP: SidekiqMiddleware::Shutdown subprocess killer
Why
We sometimes initiate a sidekiq shutdown due to excessive memory usage or gitaly unavailability. Unlike when using ^C to shutdown Sidekiq this sometimes results in processes being left running instead of properly terminating them.
What
Kills direct subprocesses during shutdown.
Uses sys-proctable to retrieve cross-platform process information and uses pgid to find subprocesses in the same process group. These are then sent a SIGTERM just before we force kill the main Sidekiq process.
Steps to reproduce
See https://gitlab.com/gitlab-org/gitlab-ce/issues/40396#note_72179341 where I've also uploaded a video in case we need to manually reverify this or implement more complicated subprocess killing in the future.
Acceptance criteria
-
Changelog entry added, if necessary -
Documentation created/updated -
Tests added for this feature/bug - Review
-
Has been reviewed by Backend
-
Related
Edited by James Edwards-Jones