Add the health_check gem end expose the health_check route

Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/7887

This adds a health_check controller on /health_check (.html, .json ,.xml) and health_check/checktype (.html, .json, .xml) for example health_check/cache.json By default it checks the database, and cache.

The endpoint is only accessible if you pass a valid health check access token. A new page in the admin section titled Heath Check has been added in order to show the current token and allow you to reset it.

Should this route be restricted to admins?

@stanhu We want service probes to be able to hit it. For instance any orchestration in charge of the box you are running on, like on AWS or OpenShift.

We could still restrict access via the private_token query string or PRIVATE-TOKEN header.

@stanhu Is there sensitive information here? Since that URL and token will be in some other system's configuration, I'd rather not use an actual admin's private token.

@DouweM The information shared by health_gem seems innocent enough, but as an admin, I'm not sure if I want any user to be able to be able to probe the state of the internals of my GitLab installation.

Good point about the admin's private token. There is probably a better way to secure this endpoint (e.g. an application-specific token).

@stanhu Is the locking down something we can already do with the system level oath application? I'm not super familiar with it. I know we could register one during reconfigure in omnibus like we do for mattermost. Then have the health_check controller accept access with an admin session, or a valid system level application secret?

as an admin, I'm not sure if I want any user to be able to be able to probe the state of the internals of my GitLab installation.

@stanhu Wouldn't these kind of health issues be obvious to the user anyway with failing requests?

Added 1 commit:

• d3a5f034 - Set the standard health_check options to include the cache check.

@DouweM The gem will end up leaking exception messages for the database, cache, and email settings. That seems like more information than we might want everyone knowing.

@twk3 Ah! I had no idea it would expose that much info. In that case, it should definitely be protected.

I don't think an OAuth application would help here. A app-level secret specific for this end point would likely be safest.

@DouweM you mean like adding another token to application settings, similar to what we do for ci runner registration?

@twk3 Yeah

Added 1 commit:

• 0f7f1c88 - Add health_check access token, and enforce on the health_check endpoint

Unmarked this merge request as a Work In Progress

Reassigned to @twk3

Added the access token for the health check, and a page in the admin section in order to reset it. Still needs docs and tests.

Added 1 commit:

• b3fe63b5 - Add back in the render_404 method on the HealthCheckController, our applicationc…

Added 528 commits:

• b3fe63b5...98d8e3fe - 525 commits from branch master
• 67dc3b9c - Add the health_check gem end expose the health_check route
• 9898f9b4 - Set the standard health_check options to include the cache check.
• 160ef66d - Add health_check access token, and enforce on the health_check endpoint

Milestone changed to 8.8

Added 1 commit:

• 0e0caf4d - Add tests for the health check feature