Add ratio metric type to the aggregation framework

What does this MR do and why?

Adds a ratio metric type to the ClickHouse aggregation framework: Gitlab::Database::Aggregation::ClickHouse::Ratio in lib/gitlab/database/aggregation/click_house/ratio.rb, registered as ratio: in Engine.metrics_mapping in lib/gitlab/database/aggregation/click_house/engine.rb. It emits numerator_agg(numerator) / nullIf(denominator_agg(denominator), 0).

Implements https://gitlab.com/gitlab-org/gitlab/-/issues/617052

The problem

Dashboard figures like "average credits spent per Duo-created MR" could only be built by fetching two totals and dividing them in the caller. Two widgets could divide the same two numbers differently, and a zero denominator surfaced as inf/nan rather than "no data".

Every existing metric type emits exactly one outer aggregate over its own inner column, and no metric can reference another metric's column. A ratio of two aggregates was therefore not merely unimplemented, it was structurally impossible.

mean is not a substitute. Averaging a row-level division gives a mean of ratios, a statistically different number from the ratio of sums:

Rows Mean of ratios Ratio of sums
(credits 10, MRs 1), (credits 10, MRs 9) avg(10/1, 10/9) = 5.56 (10+10)/(1+9) = 2.00

The right-hand column is the correct answer to "average credits per MR". The two diverge whenever the denominator varies across rows.

The design

Each metric definition owns exactly two expression slots: a primary one, and a secondary one normally used by the if: kwarg.

Type Primary slot Secondary slot Outer SQL
count value (optional) condition count(*) or countIf(sec = 1)
sum value condition sum(pri) or sumIf(pri, sec = 1)
mean value condition avg(pri) or avgIf(pri, sec = 1)
rate condition condition countIf(pri = 1) / countIf(sec = 1)
ratio (new) value value agg(pri) / nullIf(agg(sec), 0)

ratio is the first type to use both slots for values, which is what yields aggregate ÷ aggregate. Because the denominator occupies the secondary slot, ratio cannot also accept if:, and raises ArgumentError if given one rather than silently returning a wrong number.

A consequence worth noting: no changes were needed to the engine, query builder, GraphQL resolvers, types or enums, and no migration was required. Dimensions, filters, parameters, authorize:, formatter: and dotted identifiers all work by inheritance. Registration is a single line.

Three further decisions:

  • Aggregate functions are an allowlist of symbols, exactly the four the issue names: sum, count, uniqExact, avg. Validated in the constructor, so a typo raises at class-definition time. Free-form SQL is rejected, which keeps the DSL declarative.
  • nullIf(denominator, 0) is required because ClickHouse returns inf/nan on float division by zero rather than raising. Only the denominator is guarded; a numerator aggregating to zero is a real zero.
  • The declared type is restricted to :float, matching the DSL in the issue. An :integer ratio would be exposed as a GraphQL Int, whose result coercion truncates 0.667 to 0 without raising, so a wrong type is rejected at definition time rather than silently losing every value.

Both numerator and denominator must return an expression for every parameter combination. A nil from either side is caught during plan validation and returned as an error response, rather than raising during query construction where it would surface as a 500.

Test evidence

Suite Result
spec/lib/gitlab/database/aggregation/click_house/ratio_spec.rb, against live ClickHouse 28 examples, 0 failures
Full aggregation framework suite, plus the GraphQL type spec 492 examples, 0 failures
Engine consumers, ee/spec/models/analytics/aggregation_engines/ 298 examples, 0 failures
DAP GraphQL request spec, ee/spec/requests/api/graphql/analytics/duo_workflows_spec.rb 59 examples, 0 failures

The ratio spec covers default sum/sum, per-side aggregates, a zero denominator returning NULL, composition with a dimension and with a filter, ordering, parameterized expressions, a nil expression on either side, a non-float declared type, each ArgumentError path, and the inherited authorize: and formatter: options. The authorize: cases matter because credits_used on the DuoWorkflows engine carries authorize: :read_agent_artifacts, so any real credits ratio will depend on them: a denied ability drops the metric from the payload rather than returning its value.

Notes for reviewers

  1. Acceptance-criteria deviation. The issue asks for GraphQL exposure verified "in a request spec". A real HTTP request spec would require declaring a ratio metric on a mounted engine, which the issue's own out-of-scope list assigns to follow-up work, and which would churn the checked-in GraphQL schema dump. Instead, coverage was added to spec/graphql/types/analytics/aggregation/engine_response_type_spec.rb, which already builds a throwaway engine with its own metrics_mapping, asserting a ratio surfaces as a nullable Float field. Please confirm this substitution is acceptable.
  2. Pre-existing defect found during this work. Rate performs the same division with no nullIf guard, so it returns inf/nan on a zero denominator across 7 call sites in 5 shipped engines. Filed separately as #629823 (closed) and fixed in !256642 (merged), which is independent of this MR and can merge in either order.
  3. count semantics. count maps to count(col), which counts non-NULL values rather than rows, so count(*) is not directly expressible. Measured on a column that is NULL for two of three rows, the same numerator gives 60.0 with count(col) against 20.0 with a row count. count is kept because the issue names it, and the caveat plus the sql('1') workaround are documented.
  4. No changelog entry: the type is framework-internal and unreachable until a concrete engine declares a ratio metric, which is follow-up work.

How to set up and validate locally

  1. Run the ratio spec. It is tagged :click_house and needs a running ClickHouse instance:

    bundle exec rspec spec/lib/gitlab/database/aggregation/click_house/ratio_spec.rb
  2. Declare a ratio metric inside a metrics do ... end block on any ClickHouse engine. Both expressions are row-level values; the aggregate is applied by the outer query, not inside the expression:

    metrics do
      ratio :credits_per_created_mr,
        numerator: ->(_params) { sql('credits_used') },
        denominator: ->(_params) { sql('length(created_merge_request_ids)') },
        description: 'Average credits spent per merge request created by a flow'
    end
  3. Query it by its identifier, :credits_per_created_mr_ratio. The outer query emits:

    sum(credits_used) / nullIf(sum(length(created_merge_request_ids)), 0)
  4. To use different aggregates per side, pass numerator_agg: and denominator_agg:. For example denominator_agg: :uniqExact over user_id gives a per-user figure.

There is no UI to exercise yet: no engine declares a ratio metric, so nothing reaches GraphQL or a dashboard until the follow-up work lands.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Chandra Saripaka

Merge request reports

Loading
Loading