Catch unhandled exceptions in health checks
What does this MR do?
Fixes #23927 (closed)
Right now when redis is down, the redis readiness check raises Redis::CannotConnectError (Error connecting to Redis on localhost:6380 (Errno::ECONNREFUSED))
This unhandled exception then makes the readiness endpoint return a static HTML error page when run in production mode.
Instead, we should catch unhandled exceptions at the check level and return a failed check result.
With this MR, the endpoint now returns the expected result:
{
"db_check":{
"status":"ok"
},
"redis_check":{
"status":"failed",
"message":"unexpected Redis check result: Error connecting to Redis on localhost:6380 (Errno::ECONNREFUSED)"
},
"cache_check":{
"status":"failed",
"message":"unexpected Cache check result: Error connecting to Redis on localhost:6380 (Errno::ECONNREFUSED)"
},
"queues_check":{
"status":"failed",
"message":"unexpected Queues check result: Error connecting to Redis on localhost:6381 (Errno::ECONNREFUSED)"
},
"shared_state_check":{
"status":"failed",
"message":"unexpected Shared_state check result: Error connecting to Redis on localhost:6382 (Errno::ECONNREFUSED)"
},
"gitaly_check":{
"status":"ok",
"labels":{
"shard":"default"
}
}
}
Screenshots
Does this MR meet the acceptance criteria?
Conformity
-
Changelog entry -
Documentation created/updated or follow-up review issue created -
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides -
Separation of EE specific content
Performance and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. -
Tested in all supported browsers
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
-
Label as security and @ mention @gitlab-com/gl-security/appsec
-
The MR includes necessary changes to maintain consistency between UI, API, email, or other methods -
Security reports checked/validated by a reviewer from the AppSec team