Skip to content

Trap kill signal and call cleanup/stop

Ash McKenzie requested to merge ashmckenzie/trap-kill-and-cleanup-for-sidekiq into master

What does this MR do and why?

In the context of GDK, when running gdk stop or gdk stop rails-background-jobs, the TERM signal is sent to the bin/background_jobs script. Without this change, bin/background_jobs is immediately killed but the sidekiq / sidekiq-cluster processes remain, as orphans/zombies.

This MR traps when the TERM signal is sent to bin/background_jobs and runs the cleanup() function before exiting. cleanup() currently calls the stop() function to ensure a proper shutdown for sidekiq / sidekiq-cluster.

Closes: gitlab-development-kit#1344 (closed)

How to set up and validate locally

  1. Ensure you're sitting in your GDK root directory and you're up-to-date:

    cd <your-GDK-root> && git fetch && git checkout main

  2. Ensure a clean slate:

    gdk stop && gdk kill && pkill -9 sidekiq

    You may need to run ☝ a few times to ensure there's no sidekiq or sidekiq-cluster processes.

  3. Ensure you don't see sidekiq running:

    ps -ef | grep sidekiq

  4. Ensure you have the bare minimum 'DB' services and rails-background-jobs running:

    gdk start db rails-background-jobs

  5. Ensure you can see sidekiq running:

    ps -ef | grep sidekiq

  6. Take note of the output from ☝ and save it somewhere for reference.

  7. In your GDK_ROOT/gitlab/ directory, check out the ashmckenzie/trap-kill-and-cleanup-for-sidekiq branch:

    cd <your-GDK-root>/gitlab/ && git fetch && git checkout ashmckenzie/trap-kill-and-cleanup-for-sidekiq

  8. Stop rails-background-jobs:

    gdk stop rails-background-jobs

  9. Ensure you don't see sidekiq running (give it a few seconds):

    ps -ef | grep sidekiq

Contrast the above steps using origin/master in your GDK_ROOT/gitlab/ and you'll see 'orphan' sidekiq / sidekiq-cluster processes. An orphan/zombie process is one that has 'lost' its parent process and is identifiable by the parent process ID (PPID) being 1.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Ash McKenzie

Merge request reports