Readiness / liveness probe return 200 on missing database connection
Related: gitlab-org/gitlab#219046 (closed)
The webservice subchart has liveness/readiness probes configured that use the
/-/readiness respectively. BUT, those endpoints do not fail when the database connection has been dropped or cannot be made, and thus keep those pods available when they cannot effectively serve traffic.
Steps to reproduce
Run the chart, then reconfigure postgresql to use
max_connections=1. Note that the webservice-pods keep running and keep serving traffic, but return 500s for every database-hit.
After initial deploy, update postgresql:
postgresql: postgresqlConfiguration: maxConnections: 1
The webservice pods stay 'ready' according to the cluster and thus keep receiving traffic, but fail to handle this (database-hitting) traffic.
The webservice pods without valid database connections go to a non-ready state, not receiving more traffic from the ingresses/service.
- Chart: all
- Platform: any
- Kubernetes: any
- Helm: any