Allow prometheus in K8s cluster to be installed manually
#217859 (closed) inspires me to ask about this:
Is it feasible to allow user to install Prometheus themselves then tell GitLab that where it is installed ?
- For initial iteration, they can install it in a conventional location in
gitlab-managed-apps
- They send a API call to
/api/v4/clusters/:cluster_id/prometheus/installed
to let GitLab know prometheus is installed.
Questions
- I see a lot of configuration in https://gitlab.com/gitlab-org/gitlab/-/blob/master/vendor/prometheus/values.yaml. Which config is mandatory and which are not ?
- Does GitLab update the prometheus Deployment in any way ?
- How does GitLab send a request to Prometheus ? Is this configurable ?
Proposal
I would suggest that :
- User can install Prometheus with whatever config in the
gitlab-managed-apps
namespace, but the service name has to matchhttps://gitlab.com/gitlab-org/gitlab/-/blob/e731618eaf8d7425ade89984550d60d823719f79/app/models/clusters/applications/prometheus.rb#L55
- There is a way for user to hint to GitLab (webhook?) that Prometheus is now installed. GitLab may open a connection to the Prometheus service to verify Prometheus is indeed installed correctly
- Optional: user can install with suggested
scrape_configs
fromhttps://gitlab.com/gitlab-org/gitlab/-/blob/master/vendor/prometheus/values.yaml
. If thescrape_configs
are not present, then GitLab should gracefully fail - Alerts: User will have setup alerting manually as described in https://docs.gitlab.com/ee/operations/metrics/alerts.html#external-prometheus-instances
Edited by Thong Kuah