Sidekiq Pods take a seemingly long time to transition to state Ready
It seems that the Readiness probes take a long time to report that the sidekiq service is ready, in some cases over 1 minute. This is in addition to the already lengthy start times of the Pod. Utilize this issue to investigate the cause of Pods taking such a long time. Determine if there are any improvements that can be made. This has a direct impact on the ability to quickly scale and quickly roll out new versions of the application.
Steps to reproduce
- Kill a running sidekiq Pod
- Watch the replacement Pod take over 1 minute to become ready.
Long transition times between init and Ready. 71 seconds the example noted below.
Start up times less than 1 minute.
- Cloud: GKE
- Kubernetes: (
- Helm: (
- Server: n/a
Events: Type Reason Age From Message ---- ------ ---- ---- ------- <SNIP> Normal Created 77s kubelet, gke-pre-gitlab-gke-node-pool-0-715a725e-kmxd Created container sidekiq Normal Started 76s kubelet, gke-pre-gitlab-gke-node-pool-0-715a725e-kmxd Started container sidekiq Warning Unhealthy 36s kubelet, gke-pre-gitlab-gke-node-pool-0-715a725e-kmxd Liveness probe failed: Get http://10.235.3.15:8083/liveness: dial tcp 10.235.3.15:8083: connect: connection refused Warning Unhealthy 31s (x5 over 71s) kubelet, gke-pre-gitlab-gke-node-pool-0-715a725e-kmxd Readiness probe failed: Get http://10.235.3.15:8083/readiness: dial tcp 10.235.3.15:8083: connect: connection refused