Bundle Grafana and official dashboard with Omnibus for self-monitoring

Summary

Currently, we require users to perform manual tasks in order to view the instance metrics they collect with Prometheus. Namely, to download and install Grafana, configure the Prometheus data source, and then add the GitLab dashboards.

We should make this easier.

Proposal

We should provide an option to automate these actions, by bundling Grafana and setting up the dashboards and Prometheus data source. There is an existing cookbook which can help: https://github.com/sous-chefs/grafana

We should consider the core MVC as the following:

1. Optionally install Grafana, configure the local Prometheus datasource and the Omnibus dashboards.
2. Grafana would be available read-only, and simply display the GitLab dashboard.

If we have time, we can work on the next two iterations in separate MR's. This will ensure we at minimum ship the initial core functionality.

1. It would be great to allow configuration of the Grafana root user. This way users could add/edit dashboards as they see fit.
2. Next, we should enable OAuth authentication with GitLab out of the box and offer at /-/grafana. If we can do this, we can consider enabling by default.

---

We should consider also consider the relevant configuration settings:

• prometheus_monitoring['enable'] = false - should disable Grafana like it does the rest of our Prometheus related services.
• We should also have a set of grafana[] flags, similar to what we have for prometheus:
  • enable - default to false for now
  • username/group/etc - can we re-use the Prometheus user?
  • listen_address - default to 0.0.0.0 so end users can reach it.

References

Related to https://gitlab.com/gitlab-org/gitlab-ce/issues/35062 (and lesser so #2999 (closed)).