Refactor Sidekiq Memory Killer
Issues we should solve
Logging that we are above soft limit where current rss is actually below
When Sidekiq memory killer
detect that process RSS is above soft limit, it logs Sidekiq worker RSS out of range
in sidekiq structured log with the reason
:
Although we reached above the soft limit, it could happen that current_rss dropped in the meantime and that we logged the value that is actually lower than the soft limit
Specs are testing implementation details
Specs are testing private methods and implementation details which makes it difficult to change logic
The phase metric is confusing
phase
conflates the concepts of "what phase of the lifecycle are we in?" (running, shutting down, etc) and "what is our current state?" (memory usage is fine, we are over the soft limit, we are over the hard limit).
Allow Sidekiq memory killer to run and collect data without actually killing the process
Edited by Nikola Milojevic