Add ratio metric type to the aggregation framework
What does this MR do and why?
Adds a ratio metric type to the ClickHouse aggregation framework: Gitlab::Database::Aggregation::ClickHouse::Ratio in lib/gitlab/database/aggregation/click_house/ratio.rb, registered as ratio: in Engine.metrics_mapping in lib/gitlab/database/aggregation/click_house/engine.rb. It emits numerator_agg(numerator) / nullIf(denominator_agg(denominator), 0).
Implements https://gitlab.com/gitlab-org/gitlab/-/issues/617052
The problem
Dashboard figures like "average credits spent per Duo-created MR" could only be built by fetching two totals and dividing them in the caller. Two widgets could divide the same two numbers differently, and a zero denominator surfaced as inf/nan rather than "no data".
Every existing metric type emits exactly one outer aggregate over its own inner column, and no metric can reference another metric's column. A ratio of two aggregates was therefore not merely unimplemented, it was structurally impossible.
mean is not a substitute. Averaging a row-level division gives a mean of ratios, a statistically different number from the ratio of sums:
| Rows | Mean of ratios | Ratio of sums |
|---|---|---|
| (credits 10, MRs 1), (credits 10, MRs 9) | avg(10/1, 10/9) = 5.56 |
(10+10)/(1+9) = 2.00 |
The right-hand column is the correct answer to "average credits per MR". The two diverge whenever the denominator varies across rows.
The design
Each metric definition owns exactly two expression slots: a primary one, and a secondary one normally used by the if: kwarg.
| Type | Primary slot | Secondary slot | Outer SQL |
|---|---|---|---|
count |
value (optional) | condition | count(*) or countIf(sec = 1) |
sum |
value | condition | sum(pri) or sumIf(pri, sec = 1) |
mean |
value | condition | avg(pri) or avgIf(pri, sec = 1) |
rate |
condition | condition | countIf(pri = 1) / countIf(sec = 1) |
ratio (new) |
value | value | agg(pri) / nullIf(agg(sec), 0) |
ratio is the first type to use both slots for values, which is what yields aggregate ÷ aggregate. Because the denominator occupies the secondary slot, ratio cannot also accept if:, and raises ArgumentError if given one rather than silently returning a wrong number.
A consequence worth noting: no changes were needed to the engine, query builder, GraphQL resolvers, types or enums, and no migration was required. Dimensions, filters, parameters, authorize:, formatter: and dotted identifiers all work by inheritance. Registration is a single line.
Three further decisions:
- Aggregate functions are an allowlist of symbols, exactly the four the issue names:
sum,count,uniqExact,avg. Validated in the constructor, so a typo raises at class-definition time. Free-form SQL is rejected, which keeps the DSL declarative. nullIf(denominator, 0)is required because ClickHouse returnsinf/nanon float division by zero rather than raising. Only the denominator is guarded; a numerator aggregating to zero is a real zero.- The declared type is restricted to
:float, matching the DSL in the issue. An:integerratio would be exposed as a GraphQLInt, whose result coercion truncates0.667to0without raising, so a wrong type is rejected at definition time rather than silently losing every value.
Both numerator and denominator must return an expression for every parameter combination. A nil from either side is caught during plan validation and returned as an error response, rather than raising during query construction where it would surface as a 500.
Test evidence
| Suite | Result |
|---|---|
spec/lib/gitlab/database/aggregation/click_house/ratio_spec.rb, against live ClickHouse |
28 examples, 0 failures |
| Full aggregation framework suite, plus the GraphQL type spec | 492 examples, 0 failures |
Engine consumers, ee/spec/models/analytics/aggregation_engines/ |
298 examples, 0 failures |
DAP GraphQL request spec, ee/spec/requests/api/graphql/analytics/duo_workflows_spec.rb |
59 examples, 0 failures |
The ratio spec covers default sum/sum, per-side aggregates, a zero denominator returning NULL, composition with a dimension and with a filter, ordering, parameterized expressions, a nil expression on either side, a non-float declared type, each ArgumentError path, and the inherited authorize: and formatter: options. The authorize: cases matter because credits_used on the DuoWorkflows engine carries authorize: :read_agent_artifacts, so any real credits ratio will depend on them: a denied ability drops the metric from the payload rather than returning its value.
Notes for reviewers
- Acceptance-criteria deviation. The issue asks for GraphQL exposure verified "in a request spec". A real HTTP request spec would require declaring a ratio metric on a mounted engine, which the issue's own out-of-scope list assigns to follow-up work, and which would churn the checked-in GraphQL schema dump. Instead, coverage was added to
spec/graphql/types/analytics/aggregation/engine_response_type_spec.rb, which already builds a throwaway engine with its ownmetrics_mapping, asserting a ratio surfaces as a nullableFloatfield. Please confirm this substitution is acceptable. - Pre-existing defect found during this work.
Rateperforms the same division with nonullIfguard, so it returnsinf/nanon a zero denominator across 7 call sites in 5 shipped engines. Filed separately as #629823 (closed) and fixed in !256642 (merged), which is independent of this MR and can merge in either order. countsemantics.countmaps tocount(col), which counts non-NULLvalues rather than rows, socount(*)is not directly expressible. Measured on a column that isNULLfor two of three rows, the same numerator gives60.0withcount(col)against20.0with a row count.countis kept because the issue names it, and the caveat plus thesql('1')workaround are documented.- No changelog entry: the type is framework-internal and unreachable until a concrete engine declares a ratio metric, which is follow-up work.
How to set up and validate locally
-
Run the ratio spec. It is tagged
:click_houseand needs a running ClickHouse instance:bundle exec rspec spec/lib/gitlab/database/aggregation/click_house/ratio_spec.rb -
Declare a ratio metric inside a
metrics do ... endblock on any ClickHouse engine. Both expressions are row-level values; the aggregate is applied by the outer query, not inside the expression:metrics do ratio :credits_per_created_mr, numerator: ->(_params) { sql('credits_used') }, denominator: ->(_params) { sql('length(created_merge_request_ids)') }, description: 'Average credits spent per merge request created by a flow' end -
Query it by its identifier,
:credits_per_created_mr_ratio. The outer query emits:sum(credits_used) / nullIf(sum(length(created_merge_request_ids)), 0) -
To use different aggregates per side, pass
numerator_agg:anddenominator_agg:. For exampledenominator_agg: :uniqExactoveruser_idgives a per-user figure.
There is no UI to exercise yet: no engine declares a ratio metric, so nothing reaches GraphQL or a dashboard until the follow-up work lands.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.