Skip to content

Add timeout to health check database inserts

Sami Hiltunen requested to merge smh-health-check-timeout into master

HealthManager currently does not apply a timeout to its database updates. This can cause hard to diagnose issues where the health checks are considered immediately outdated if the inserting takes longer than the failover timeout. Currently the failover timeout is 10 seconds, so any inserts that take longer than 10 seconds are immediately considered unhealthy when they complete. The inserts really shouldn't take long. This commit adds a timeout for the inserts so database writes taking too long end up being canceled and thus logged as an error. Logging the error helps identify the problem when Gitalys are responding to health checks but Praefect thinks they are unhealthy.

Edited by Sami Hiltunen

Merge request reports