Resolve "AgentPlatformSessions AE: project_id dimension for per-project flow/chat counts"

What does this MR do and why?

Adds a project_id dimension and filter to Analytics::AggregationEngines::AgentPlatformSessions, so the engineering intelligence dashboard can get Duo Agent Platform flows per project and chats per project (flowType: ["chat"]) in a single request.

Before this, a per-project breakdown needed one scoped request per project, and AggregationScopeInput accepts at most 20 sources.

No schema work

agent_platform_sessions already has a real project_id UInt64 column, populated by agent_platform_sessions_mv from JSONExtractUInt(extras, 'project_id'), and it is already present in db/click_house/schema_cache/main/agent_platform_sessions.yml. So there is no ClickHouse migration, no db/click_house/main.sql change and no schema-migration marker file. The GraphQL field and argument are generated from the engine declaration, so there are no resolver or type changes either.

Why the dimension goes through a transient

project_id sits outside the table sort key (namespace_path, user_id, session_id, flow_type). Declared as a plain column it would join the inner GROUP BY ALL and become part of the per-session grouping key. Read through any(project_id) it is an aggregate, so it stays out of the inner group-by and the inner query remains one row per session regardless of how AggregatingMergeTree has merged the underlying parts.

This is safe because a session belongs to exactly one project: both emitters (CreateWorkflowService and UpdateWorkflowStatusService) go through Ai::DuoWorkflows::Concerns::WorkflowEventTracking#track_workflow_event with project: workflow.project from the same Workflow record.

Resulting SQL:

SELECT `ch_aggregation_inner_query`.`aeq_project_id` AS aeq_project_id, COUNT(*) AS aeq_total_count
FROM (
  SELECT any(project_id) AS aeq_project_id,
    `agent_platform_sessions`.`namespace_path`, `agent_platform_sessions`.`user_id`,
    `agent_platform_sessions`.`session_id`, `agent_platform_sessions`.`flow_type`
  FROM `agent_platform_sessions`
  WHERE `agent_platform_sessions`.`project_id` IN (7)
  GROUP BY ALL
) `ch_aggregation_inner_query`
GROUP BY ALL
ORDER BY aeq_total_count DESC

Sessions without a project

JSONExtractUInt yields 0 when a session is not scoped to a project (namespace-level sessions). The dimension formats 0 to nil, and the filter accepts "0" or "none" so those sessions can be selected deliberately.

Note the limitation: exact_match only emits IN, so namespace-level sessions can be selected but not excluded without enumerating every project. Raised on the issue.

Drive-by fix: association batch loading

The first commit fixes a latent bug that this change exposes. BatchLoader keys a batch on the block source location plus an explicit key. Every association dimension is defined from the same place in declare_association_field and passed no key, so all of them shared one batch. Whichever dimension resolved first decided the model for the whole group, and ids belonging to the other dimensions were looked up against that model and came back nil.

No engine had two association dimensions before, so nothing surfaced it. With user and project on the same engine, project ids were being looked up against User. Each dimension now batches under its own key.

Note for reviewers

project_id is not in the table sort key, so the WHERE project_id IN (...) filter cannot use the primary key index. The startsWith(namespace_path, ...) scope condition is what does the pruning.

How to set up and validate locally

  1. Start ClickHouse: gdk start clickhouse
  2. Run the specs:
    bundle exec rspec ee/spec/models/analytics/aggregation_engines/agent_platform_sessions_spec.rb
    bundle exec rspec ee/spec/requests/api/graphql/analytics/ai_analytics/agent_platform_sessions_spec.rb
  3. In GraphiQL, flows per project (top 20):
    {
      group(fullPath: "gitlab-org") {
        analytics {
          agentPlatformSessions(createdEventAtFrom: "2026-01-01T00:00:00Z", createdEventAtTo: "2026-08-01T00:00:00Z") {
            aggregated(orderBy: [{ identifier: "total_count", direction: DESC }], first: 20) {
              count
              nodes { dimensions { project { id fullPath } } totalCount }
            }
          }
        }
      }
    }
  4. Add flowType: ["chat"] for chats per project, or projectId: ["0"] to select namespace-level sessions.

References

Related to https://gitlab.com/gitlab-org/gitlab/-/issues/608198

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Brandon Labuschagne

Merge request reports

Loading
Loading