Consul does not have liveness probes
Details
- Point of contact for this request: @skarbek
- If a call is needed, what is the proposed date and time of the call: N/A
- Additional call details (format, type of call): N/A
SRE Support Needed
ServiceConsul lacks liveness probes. This can result in Pods that are running, but could be unhealthy, which lead to application failures due to their reliance on consul
's DNS resolution for our database endpoints. We recently suffered an incident where a consul
Pod had been alive for over 30+ hours and was never restarted. The resolution of the incident was to delete said Pod.
Leverage this issue to determine the implementation of liveness probes in consul.
Reference: production#17780 (closed)
Edited by Milad Irannejad