Use ClickHouse for MR throughput on Duo and SDLC trends dashboard

Abhinaba Ghoshrequested to merge
627208-mr-throughput-clickhouse-data-source into master

What does this MR do and why?

On the Duo & SDLC trends dashboard, Merge request throughput is the only metric computed live from PostgreSQL (Group.mergeRequests(includeSubgroups: true).count). For large group hierarchies this count exceeds the 15-second statement timeout on GitLab.com, and the metric renders as an indefinite skeleton for every viewer.

This MR reads the metric from the Siphon-replicated ClickHouse merge_requests table through the existing analytics.mergeRequests aggregation engine instead. Verified on production for gitlab-org (March 2026): the PostgreSQL count is killed at 15s, the ClickHouse aggregation returns 24,059 merged MRs in under 2s.

The ClickHouse path applies only where the data exists:

  • new siphon_clickhouse_analytics SaaS feature — Siphon runs on GitLab.com only, so self-managed and Dedicated instances with ClickHouse enabled keep the PostgreSQL count (their table would be empty)
  • mr_throughput_clickhouse_data_source feature flag (gitlab_com_derisk, group actor) for a customer-first rollout

References

Screenshots or screen recordings

No visual changes. Before: the Merge request throughput row stays on skeleton loaders for large groups. After: the row loads.

How to set up and validate locally

Requires GDK with clickhouse and siphon enabled (the merge_requests ClickHouse table must be populated).

  1. Simulate SaaS: export GITLAB_SIMULATE_SAAS=1, restart Rails.

  2. In rails console:

    ApplicationSetting.current.update!(use_clickhouse_for_analytics: true)
Feature.enable(:mr_throughput_clickhouse_data_source)

  3. Visit a group's Analyze > Analytics dashboards > GitLab Duo and SDLC trends.

  4. Verify the Merge request throughput row loads and the network tab shows the mergeRequestsThroughputClickhouse operation.

  5. Disable the flag and reload to verify the PostgreSQL fallback (mergeRequests operation) still renders the row.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist.

Merge request reports