prometheus_metrics.md 1.68 KB
Newer Older
1 2 3 4 5 6 7 8
# Working with Prometheus Metrics

## Adding to the library

We strive to support the 2-4 most important metrics for each common system service that supports Prometheus. If you are looking for support for a particular exporter which has not yet been added to the library, additions can be made [to the `common_metrics.yml`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/config/prometheus/common_metrics.yml) file.

### Query identifier

Kamil Trzciński's avatar
Kamil Trzciński committed
9
The requirement for adding a new metric is to make each query to have an unique identifier which is used to update the metric later when changed:
10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

```yaml
- group: Response metrics (NGINX Ingress)
  metrics:
  - title: "Throughput"
    y_label: "Requests / Sec"
    queries:
    - id: response_metrics_nginx_ingress_throughput_status_code
      query_range: 'sum(rate(nginx_upstream_responses_total{upstream=~"%{kube_namespace}-%{ci_environment_slug}-.*"}[2m])) by (status_code)'
      unit: req / sec
      label: Status Code
```

### Update existing metrics

After you add or change existing _common_ metric you have to create a new database migration that will query and update all existing metrics.

Kamil Trzciński's avatar
Kamil Trzciński committed
27 28 29 30
NOTE: **Note:**
If a query metric (which is identified by `id:`) is removed it will not be removed from database by default.
You might want to add additional database migration that makes a decision what to do with removed one.
For example: you might be interested in migrating all dependent data to a different metric.
31 32

```ruby
33
class ImportCommonMetrics < ActiveRecord::Migration[4.2]
Kamil Trzciński's avatar
Kamil Trzciński committed
34 35
  include Gitlab::Database::MigrationHelpers

36 37 38
  DOWNTIME = false

  def up
39
    ::Gitlab::DatabaseImporters::CommonMetrics::Importer.new.execute
40 41 42 43 44 45 46
  end

  def down
    # no-op
  end
end
```