Remove legacy memory killer
The motivation
Followup https://gitlab.com/gitlab-com/gl-infra/infrastructure/-/issues/10930#note_386967266. We should consider removing the Sidekiq legacy memory killer to simplify the configuration management.
We have two Sidekiq memory killers: legacy memory killer(lib/gitlab/sidekiq_middleware/memory_killer.rb) and daemon memroy killer(lib/gitlab/sidekiq_daemon/memory_killer.rb).
The default one is still the legacy memory killer. From Configuring the MemoryKiller
SIDEKIQ_DAEMON_MEMORY_KILLER: defaults to 0. When set to 1, the MemoryKiller works in daemon mode. Otherwise, the MemoryKiller works in legacy mode.
We prefer to use the Daemon memory killer to replace the legacy one. The daemon memory killer has some advantages: !41847 (comment 412020465)
Before GitLab 13.4, the daemon memory killer was NOT used in production
- Gitlab.com
- Kubernetes: not using
any
Sidekiq memory killer, per #232756 (comment 387454442) - VM: was using the legacy memory killer, per !41847 (comment 412155912)
- Kubernetes: not using
- Omnibus: is using
legacy memory killer
as default
To be safe, we take three steps
-
make daemon memory killer be the default for Sidekiq - Done by MR !41847 (merged). So it started from version 13.4
- Gitlab.com Kubernetes Sidekiq urgent queues started to enable the daemon memory killer from 13.4, and it is working as expected, per gitlab-com/gl-infra/delivery#1084 (comment 416000721).
-
monitoring and fix the daemon memory killer bugs reported(how long? maybe 2 releases?) - thread safety issue can't add a new key into hash during iteration, fixed by MR !42468 (merged)
-
remove the legacy memory killer
Edited by Qingyu Zhao