[Meta] Prometheus Environment monitoring for Kubernetes based auto-deploys

Description

Kubernetes offers basic system monitoring capabilities out of the box via Prometheus. If GitLab could consume these metrics, a turnkey monitoring solution could be offered to customers with little to no effort on their side.

Further, by integrating insight about metrics into workflows inside of GitLab like Environments and Merge Requests, we could offer value they could not get from a monitoring system alone.

Proposal

We can take advantage of the fact that GitLab already supports automatic deployments to Kubernetes in a well defined way. This allows the metrics which are reported by Kubernetes to be accurately matched with their corresponding pairs in GitLab.

With the matching in place, we can identify metrics for a specific environment and then display these on the Environment detail page so a user can see how a particular environment is performing.

The work items (prioritized) are:

• #26910 (closed) - Display charts on Environments page for a specific metrics
• #26912 (closed) - Configure GitLab's Prometheus server to monitor Kubernetes
• #26944 (closed) - Display Prometheus sparkline in Merge Request
• #26911 (closed) - Support Prometheus running on its own container / server
• #26914 (closed) - Include deploy history on metric chart for Environments page

We have already mocked up a possible version of this for the Prometheus demo, which we can use as a starting point.