Skip to content

fix(stgsub): fix external labels for prometheus

Steve Xuereb requested to merge fix/stgsub-labels into master

Background

Currently we have the following prometheus configuration:

global:
  scrape_interval: 15s
  scrape_timeout: 10s
  evaluation_interval: 15s
  external_labels:
    cluster: stgsub-customers-gke
    env: stgsub
    environment: stgsub
    monitor: default
    prometheus: monitoring/gitlab-monitoring-promethe-prometheus
    prometheus_replica: prometheus-gitlab-monitoring-promethe-prometheus-0
    provider: gcp
    region: us-east1

Notice how env="stgsub" and environment="stgsub". This doesn't follow our label taxanomy in https://gitlab.com/gitlab-com/runbooks/-/tree/master/libsonnet/label-taxonomy where it assumes it's going to be gstg or gprd.

This results into thanos-store and thanos-sidecar metrics to be exposed with the wrong labels as we see in gitlab-com/gl-infra&696 (comment 875284133)

Solution

Specify the monitoring_env explictly for stgsub so that we set this to gstg which checks checked in values

Signed-off-by: Steve Azzopardi sazzopardi@gitlab.com

Merge request reports