Attackers can snoop on any project's prometheus metrics and environments
The following issue was reported through HackerOne.
Weakness: Insecure Direct Object Reference (IDOR) Link: https://hackerone.com/reports/406578
Details:
The Projects::Prometheus::AlertsController#create
and Projects::Prometheus::AlertsController#update
allow an attacker to provide an environment_id
and prometheus_metric_id
that doesn't belong to their project. When an alert persists that doesn't belong to their project, they can trigger it using the Projects::Prometheus::AlertsController#notify
endpoint. This will leak information about the victim's environment, project, and their Prometheus metric.
Proof of concept
For the vulnerability to be exploitable, the user requires the prometheus_alerts
feature. I manually enabled this on my GitLab EE instance by monkey patching the feature_available?
helper on the Project
model. It can also be made available by acquiring the appropriate GitLab license. To reproduce the vulnerability, follow the steps below.
As the victim
- sign in to any user
- create a project and manually enable Prometheus monitoring (can point to a host that's offline) through the Integrations section
- create a new metric for the Prometheus monitoring
- create an environment for the project
As the attacker
- sign in as a user that can create projects, make sure that the
prometheus_alerts
feature is available - create a new project
- send a
POST
request to the/:namespace/:project/prometheus/alerts.json
endpoint, as such:
Request
POST /:namespace/wow/prometheus/alerts.json HTTP/1.1
...
X-CSRF-Token: ...
X-Requested-With: XMLHttpRequest
Cookie: ...
Content-Type: application/json
Content-Length: 78
{"operator":">","threshold":1,"environment_id":1,"prometheus_metric_id":1}
- in the request above, change the
environment_id
andprometheus_metric_id
to the objects that were created by the victim, the server will return a response similar to the one below:
Response
HTTP/1.1 200 OK
Server: nginx
...
{"id":6,"title":"Metric title","query":"rate()","threshold":1.0,"operator":"\u003e","alert_path":"/:namespace/:project/prometheus/alerts/1.json?environment_id=1"}
- in this response, some confidential information can be obtained: the
title
andquery
of the metric that was referenced - now, send a
POST
request to the/:namespace/:project/prometheus/alerts/notify
endpoint, as such:
Request
POST /:namespace/:project/prometheus/alerts/notify HTTP/1.1
...
Content-Type: application/json
Cookie: ...
{"alerts":[{"labels":{"gitlab_metric_id":1}}]}
- this will send an email to the masters / owners of the
/:namespace/:project
project with the environment name and the full path of the project that has the referencedgitlab_metric_id
. Note: this information leak ONLY works when the attacker only referenced anenvironment_id
that belongs to another (private) project. It does NOT work when agitlab_metric_id
was referenced that doesn't belong to the project. This is because the asynchronous job becomes a no-op when thegitlab_metric_id
doesn't belong to the project referenced in the URL (it's like someone foresaw this issue...). Because the attacker isn't authorized to create Prometheus metrics / alerts for the project, this will never work. This endpoint does have another vulnerability though, see "Bonus vulnerability" at the bottom of the report.
What I'm currently unable to figure out is whether a (Kubernetes) cluster with Prometheus configured is provisioned with the alerts that belong to another project. If that's the case, the attacker can snoop on any metric by creating an alert that will always trigger (using lt 0
, gt 0
, and eq 0
). This may increase the severity of the vulnerability. However, I didn't set up a Prometheus configuration on Kubernetes, so I haven't confirmed this.
Caveat: the database contains a UNIQUE
constraint on the prometheus_metric_id
column. This means that an attacker can only snoop on metrics that don't have an alert set up yet. However, this doesn't apply to the environment_id
. The attacker can reference any environment as often as they want to.
Bonus vulnerability (CSRF)
The Projects::Prometheus::AlertsController#notify
endpoint disables any CSRF protection. This means that an attacker can trigger a fake alert by letting a victim visit a web page that contains the following form
(note: the attacker needs the namespace, project name, and metric ID before this can be exploited):
<form method="post" action="http://gitlab-instance/:namespace/:project/prometheus/alerts/notify">
<input name="alerts[][labels[gitlab_metric_id]]" value="1" />
<input type="submit" />
</form>
<script>document.forms[0].submit();</script>
Impact
Exploiting this vulnerability gives an attacker access to any (private) project name, environment name, metric query, and metric title, that has at least one unmonitored Prometheus metric. This may also result in the alerts being provisioned on a Kubernetes cluster with Prometheus monitoring enabled.
The CSRF vulnerability doesn't seem to have any immediate impact for GitLab, but I'd think it's rather annoying that a project owner / master were to receive false-positive alerts notifications from GitLab.
Timeline: 2018-09-07 20:48:48 +0000: @jritchey (comment) Hi @jobert ,
Thank you for submitting this report. We are currently investigating the issue. Due to our current workload, we will get back with you as soon as we can with an update.
Best regards, James
2018-09-07 23:55:57 +0000: @asaba (user assigned to bug [team-only])