Skip to content

Add Prometheus recording rule for tracking the database query Apdex

Related issue: gitlab#227305 (closed)

Depends on: gitlab!39206 (merged) for the re-introduction of gitlab_sql_duration_seconds_bucket{le="0.25"} that is used in the query.

We want to add instrumentation for our key database metrics, in particular our Database Primary Performance Indicator: Query apdex with 100ms target, 250ms tolerable.

Resulting Prometheus query:

(
  sum(rate(gitlab_sql_duration_seconds_bucket{le="0.1"}[5m])) 
  +
  sum(rate(gitlab_sql_duration_seconds_bucket{le="0.25"}[5m]))
) 
/ 
2 
/ 
(
  sum(rate(gitlab_sql_duration_seconds_count[5m])) > 0
)

The aforementioned query is pretty expensive if run for over an hour and we want to track it at a weekly basis.

In this MR we are adding a recording rule to pre-compute a new gitlab_usage_ping:query_apdex:ratio metric every 5 minutes.

We will then use that on a followup MR to run a rolling 7 day average and report the Weekly Apdex in usage ping.

Edited by Yannis Roussos

Merge request reports