Add group dimension and filter to AiUsageEvents aggregation engine

What does this MR do and why?

Adds a group_id dimension and matching exact_match filter to the Analytics::AggregationEngines::AiUsageEvents ClickHouse aggregation engine (ee/app/models/analytics/aggregation_engines/ai_usage_events.rb).

The dimension extracts the last namespace id from the namespace_path column using toUInt64OrZero(splitByChar('/', namespace_path)[-2]) and is declared with association: true, so GraphQL exposes it as a group object field resolving to Group records. This is the same mechanism already used for the user dimension, and follows the same pattern as FinishedPipelines#project_id.

The filter accepts group Global IDs via the groupId GraphQL argument.

This unblocks the "Duo adoption by group" dashboard chart: distinct AI users per group, and per-group trends when combined with the existing monthly timestamp date bucket.

Reviewers should note: events tracked in a project store the project namespace as the last path segment, so those rows return a null group. Events with no namespace use the placeholder path 0/ and also return a null group.

GraphQL reference docs were regenerated (doc/api/graphql/reference/_index.md).

References

Resolves https://gitlab.com/gitlab-org/gitlab/-/issues/605518

Screenshots or screen recordings

This is a backend and GraphQL-only change, so there are no screenshots.

How to set up and validate locally

  1. Enable ClickHouse for analytics in GDK and make sure some AI usage events exist (for example, use Duo Chat in a group and a project, or insert rows into the ai_usage_events ClickHouse table).
  2. Open the GraphQL explorer at http://gdk.test:3000/-/graphql-explorer.
  3. Run this query (adjust fullPath):
query {
  group(fullPath: "gitlab-org") {
    analytics {
      duoUsageEvents {
        aggregated {
          nodes {
            dimensions {
              group {
                id
                fullPath
              }
              timestampMonthly: timestamp(granularity: "monthly")
            }
            usersCount
            totalCount
          }
        }
      }
    }
  }
}
  1. Expect one node per (group, month) with distinct user counts. Events tracked in projects appear under a null group.
  2. To drill into a single group, add the filter argument:
query {
  group(fullPath: "gitlab-org") {
    analytics {
      duoUsageEvents(groupId: ["gid://gitlab/Group/<group-id>"]) {
        aggregated {
          nodes {
            dimensions {
              group {
                id
                fullPath
              }
            }
            usersCount
            totalCount
          }
        }
      }
    }
  }
}

Query plan

Because ai_usage_events is ordered by (namespace_path, event, timestamp, user_id), namespace_path is the primary key prefix. Every query from this engine goes through prepare_base_aggregation_scope, which always adds a startsWith(namespace_path, '<scope prefix>') condition. ClickHouse turns that into a primary key range and prunes parts and granules via binary search before any other predicate runs.

The group_id filter, toUInt64OrZero(splitByChar('/', namespace_path)[-2]), is evaluated only on the rows that survive this pruning. It cannot use the index directly, but neither can the engine's existing feature filter (a CASE expression over event), and that has not been a performance concern. The cost profile is the same: index pruning first, computed filter second, over a much smaller row set.

Representative query, filtering by group_id = 123 within scope prefix 22/:

SELECT
    toUInt64OrZero(splitByChar('/', namespace_path)[-2]) AS group_id,
    uniqExact(user_id) AS users_count
FROM ai_usage_events
WHERE startsWith(namespace_path, '22/')
  AND toUInt64OrZero(splitByChar('/', namespace_path)[-2]) IN (123)
GROUP BY group_id

EXPLAIN indexes = 1 output from a local development ClickHouse:

Expression ((Project names + Projection))
  Aggregating
    Expression (Before GROUP BY)
      Expression ((WHERE + Change column names to column identifiers))
        ReadFromMergeTree (gitlab_clickhouse_development.ai_usage_events)
        Indexes:
          MinMax
            Condition: true
            Parts: 14/14
            Granules: 30/30
          Partition
            Condition: true
            Parts: 14/14
            Granules: 30/30
          PrimaryKey
            Keys:
              namespace_path
            Condition: (namespace_path in ['22/', '220'))
            Parts: 12/14
            Granules: 13/30
            Search Algorithm: binary search
            Ranges: 12

The PrimaryKey step confirms the scope prefix condition is doing the pruning (13/30 granules, 12/14 parts) before group_id is computed, so the per-row cost applies only to the already-narrowed result set.

On robustness, an out-of-range array access in ClickHouse returns the type default, an empty string, and toUInt64OrZero maps that (and any non-numeric segment) to 0, which resolves to a null group. Malformed and placeholder paths ('', 'abc/', '0/') are covered by specs, so a bad row degrades gracefully instead of failing the query.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Brandon Labuschagne

Merge request reports

Loading
Loading