Include Duo Chat in AI governance sessions KPI

Andrew Jungrequested to merge
607565-include-chat-in-sessions into master

Stacked on !246384 (merged) (!246384 (merged)): this MR targets that branch so the diff shows only the chat-inclusion change. GitLab retargets to master automatically when !246384 (merged) merges; draft until then.

What does this MR do and why?

Makes the AI Governance Dashboard's sessions KPI count every session, including Duo Chat. The agents KPI still counts agentic instances only. Resolves https://gitlab.com/gitlab-org/gitlab/-/work_items/607565.

Today Ai::Governance::MetricsService::EXCLUDED_DEFINITIONS = %w[chat] filters both KPIs. A tile labelled "AI sessions" reports only 56% of AI sessions (chat is ~44%: ~60,981 of ~139,622 per 7 days, plans linked in the issue), and nothing on the dashboard, in the API descriptions, or in the docs says so. The audit trail card on the same dashboard already shows chat activity, and the sessions docs already say chat creates sessions. So the KPI contradicted both. Product decision on the issue (2026-08-04): sessions include Duo Chat, no toggle. Both surfaces should cover the same sessions.

The agents KPI does not change, on purpose. Internal instances are keyed on (user, container, surface). Counting chat would add one pseudo-instance per person who opened chat, and the tile would stop measuring agents. Chat conversations count as usage (sessions), not as workers (agents).

All of this sits behind the default-off ai_governance_dashboard flag (wip, no rollout issue). No audience sees the definition change yet.

Where the exclusion now lives, and how its list is derived

  • The scope-level filter (PostgreSQL) and the WHERE predicate (ClickHouse) are gone. Sessions are now plain row counts.
  • The same exclusion is applied again, but only inside the agents aggregates: COUNT(DISTINCT <key>) FILTER (WHERE ... workflow_definition NOT IN (...)) / uniqExactIf(<key>, ... AND workflow_definition NOT IN {...}). Still one query per call on both paths.
  • The constant is renamed AGENT_EXCLUDED_DEFINITIONS and now comes from Ai::FoundationalChatAgent.only_duo_chat_agent → ["chat", "agentic_chat/v1"]. agentic_chat/v1 is chat's flow-registry successor (gitlab-org#19647). Deriving the list means the guard follows that rename instead of quietly going stale. Today this changes nothing in practice: workflow_definition = 'agentic_chat/v1' has 42 rows/30d, 4 rows/7d on gitlab-production-main (user-run 2026-08-05, read off the plan's actual rows).
  • GraphQL field descriptions now spell out both choices ("including Duo Chat conversations" / "Chat conversations are not counted"). Reference docs and introspection are regenerated.

Index changes (the migrations)

Index on duo_workflows_workflows Action Why
index_duo_workflows_workflows_on_namespace_id_created_at (partial WHERE workflow_definition <> 'chat') Replaced by index_duo_workflows_workflows_on_namespace_created_at (namespace_id, created_at DESC) WHERE namespace_id IS NOT NULL A query that covers every definition can't use an index that excludes chat. Single consumer: this metrics service (scope in_namespace_hierarchy has exactly one caller). Keeping both indexes would only add write cost with no read benefit
(new) index_duo_workflows_workflows_on_project_created_at (project_id, created_at DESC) WHERE project_id IS NOT NULL Added Without it, the project/group-project branch uses the plain (project_id) index and reads a project's whole history on every probe
index_duo_workflows_workflows_project_environment_created_at (2025, partial) Untouched Owned by WorkflowsFinder, not this feature
index_duo_workflows_workflows_on_namespace_id (single) Dropped Redundant once the new composite index exists: same leading column, so it still covers fk_7fcf81369f and every namespace_id lookup
index_duo_workflows_workflows_on_project_id (single) Dropped Redundant once the new composite index exists (still covers fk_2f6398d8ee). With both drops, the table ends up with one fewer index than master (14 vs 15 in pg_indexes), back under the 15-per-table limit

Both migrations are post-deploy, use add_concurrent_index / remove_concurrent_index_by_name, and are reversible: each down migration restores the dropped indexes exactly, and this was round-tripped locally with db:migrate:down:main. The table has ~3.7M rows, so synchronous index operations are fine.

The duplicate_indexes.yml allowlist had two duo_workflows_workflows YAML keys (last key wins, so the first entry was dead). This MR merges them and removes the entry for the dropped index.

Database review

Query plans were captured with warm second runs on gitlab-production-main (session 54490), against the namespace 9970 hierarchy (gitlab-org), with literal timestamps and no CTE. Two indexes matching the migrations were created in-session (ix_tmp_ns, ix_tmp_proj), and the planner picked both in every plan below. All four warm runs hit zero disk reads.

Query Warm time Buffers Rows into aggregates Plan
Group, totals (7d) 162.5 ms 98,015 16,325 https://console.postgres.ai/gitlab/projects/gitlab-production-main/sessions/54490/commands/157276
Group, daily buckets (7d) 130.5 ms 76,495 9,134 https://console.postgres.ai/gitlab/projects/gitlab-production-main/sessions/54490/commands/157279
Group, totals (30d) 572.4 ms 300,474 83,998 https://console.postgres.ai/gitlab/projects/gitlab-production-main/sessions/54490/commands/157281
Group, daily buckets (30d) 348.3 ms 170,912 40,705 https://console.postgres.ai/gitlab/projects/gitlab-production-main/sessions/54490/commands/157283

Compared against the chat-excluded shape on master (session 54372, same hierarchy), 30-day totals ran 285.2 ms over 126,416 buffers with 75,968 rows reaching the aggregates, and 30-day daily buckets ran 232.8 ms with 40,559 rows. Including chat roughly doubles the time on the 30-day totals query. Row count into the aggregates rises only about 11 percent (75,968 to 83,998), so most of the extra time comes from buffers (126,416 to 300,474): the namespace branch now scans an unfiltered index instead of one that excluded chat. gitlab-org is a low-chat hierarchy (its traffic is mostly security and evaluation flows), so this row growth understates the instance-wide chat share of about 44 percent. It remains the busiest hierarchy whose plans can be published.

Per database review, both composites are partial on IS NOT NULL (check constraint check_73884a5839 sets exactly one of namespace_id / project_id per row, so each full index would store a never-matched entry for every row on the other side). The four queries were re-captured with partial temp indexes (session 55038); the planner picks them in every shape and the numbers sit within noise of the run above:

Query Warm time Buffers Plan
Group, totals (7d) 171.2 ms 98,218 https://console.postgres.ai/gitlab/projects/gitlab-production-main/sessions/55038/commands/158358
Group, daily buckets (7d) 139.9 ms 102,496 https://console.postgres.ai/gitlab/projects/gitlab-production-main/sessions/55038/commands/158360
Group, totals (30d) 613.0 ms 300,687 https://console.postgres.ai/gitlab/projects/gitlab-production-main/sessions/55038/commands/158364
Group, daily buckets (30d) 392.3 ms 171,127 https://console.postgres.ai/gitlab/projects/gitlab-production-main/sessions/55038/commands/158366

In the 7-day buckets plan the namespace arm switches to a merge join that range-scans the whole partial index (31k buffers against 5k for keyed probes), a plan the small partial index makes viable; time stays within noise. Migration timings below are from the partial revision's db-testing run.

For distribution context, session 54372 found 5,723 root namespaces with project-attached sessions in the 60-day window. gitlab-org is the second busiest; exactly one root exceeds it, bracketed by HAVING probes between 151,000 and 302,000 rows. At the cost measured here, about 6.8 microseconds per row, that root would run roughly 1 to 2 seconds warm on the 30-day totals query.

The 572 ms worst case is mitigated on several fronts: this field is a dashboard aggregate behind the default-off ai_governance_dashboard flag on an experiment API, called once per dashboard load with no polling. GitLab.com serves this field from ClickHouse, so the PostgreSQL path measured here matters mainly for self-managed instances, which are far smaller than gitlab-org.

Migration testing

Results from the db:gitlabcom-database-testing pipeline on GitLab.com data:

Migration Type Runtime DB size change
20260805160752 SwapDuoWorkflowsWorkflowsNamespaceCreatedAtIndex Post deploy 16.5 s -149.76 MiB
20260805160812 AddDuoWorkflowsWorkflowsProjectCreatedAtIndex Post deploy 13.6 s +37.84 MiB

The swap reclaims space because it drops both the old chat-partial index and the redundant single-column namespace_id index while adding one partial composite that only stores namespace-attached rows (a small share of the table). The earlier full-composite revision measured -18.95 MiB and +48.27 MiB; the partials save about 142 MiB net against it. No query in the migration run exceeded 15 seconds. Both migrations are post-deploy, use add_concurrent_index and remove_concurrent_index_by_name, and each down migration restores the dropped indexes exactly.

How to validate

Everything below assumes a working GDK with an Ultimate license, checked out on this branch.

1. Migrate and enable the flags

bin/rails db:migrate
# bin/rails console
Feature.enable(:ai_governance_dashboard)        # gates the aiGovernanceMetrics field and the tiles
Feature.enable(:gitlab_duo_governance_settings) # gates the settings page hosting the dashboard
# pin the PostgreSQL read path; fresh GDKs default to this, step 6 covers ClickHouse
ApplicationSetting.current.update!(use_clickhouse_for_analytics: false)

2. Seed one group with every kind of session (same console)

me = User.find_by(username: 'root')
group = Group.create!(name: 'Chat KPI demo', path: 'chat-kpi-demo',
  organization: Organizations::Organization.first)
group.add_owner(me) # read_agent_artifacts comes with Owner, which authorizes the field
project = Projects::CreateService.new(me, name: 'demo', path: 'demo', namespace_id: group.id).execute
chatter = User.where.not(id: me.id).where(user_type: :human).first

make = ->(user:, definition:, env: :ide, at: Time.current) do
  Ai::DuoWorkflows::Workflow.create!(
    project: project, user: user, goal: 'demo', workflow_definition: definition,
    environment: env, agent_privileges: [1], pre_approved_agent_privileges: [],
    created_at: at, updated_at: at)
end

# agentic work: 3 sessions that collapse onto 2 agent instances
make.call(user: me, definition: 'software_development')
make.call(user: me, definition: 'convert_to_gitlab_ci')            # same (user, project, env) as above
make.call(user: me, definition: 'software_development', env: :web) # second instance
# chat: 2 current-window conversations from another user, 1 from last week
make.call(user: chatter, definition: 'chat')
make.call(user: chatter, definition: 'agentic_chat/v1', env: :web)
make.call(user: me, definition: 'chat', at: 9.days.ago)

3. Read the KPIs through the real API (same console)

query = <<~GQL
  query { group(fullPath: "chat-kpi-demo") { aiGovernanceMetrics(timeframe: LAST_7_DAYS) {
    sessions { count previousCount } agents { count previousCount } } } }
GQL
pp GitlabSchema.execute(query, context: { current_user: me }).dig('data', 'group', 'aiGovernanceMetrics')

Expected on this branch:

{"sessions"=>{"count"=>5, "previousCount"=>1}, "agents"=>{"count"=>2, "previousCount"=>0}}

For the before, git checkout the target branch (this MR is stacked on it), restart the console and rerun step 3 with the same data: sessions read {"count"=>3, "previousCount"=>0} because all three chat rows are silently dropped; agents read the same 2 / 0.

The two current-window chat rows deliberately carry (user, environment) combinations no other row uses, so if the agents guard were broken they would mint instances and agents would read 4, not 2. agentic_chat/v1 is chat's flow-registry successor and must stay guarded like chat.

4. Same thing over HTTP (any personal access token with api scope)

curl -s -X POST "http://gdk.test:3000/api/graphql" \
  -H "Authorization: Bearer $YOUR_PAT" -H "Content-Type: application/json" \
  -d '{"query":"query { group(fullPath: \"chat-kpi-demo\") { aiGovernanceMetrics(timeframe: LAST_7_DAYS) { sessions { count previousCount } agents { count previousCount } } } }"}'

5. See it rendered: http://gdk.test:3000/groups/chat-kpi-demo/-/settings/gitlab_duo/governance (the tiles read the same field; sessions shows 5 with a +4 delta, agents 2).

6. Optional, ClickHouse read path: with GDK ClickHouse + siphon enabled, set ApplicationSetting.current.update!(use_clickhouse_for_analytics: true) and rerun step 3 once siphon has replicated the rows; the numbers are identical. The ClickHouse service specs assert the same fixture math if you prefer not to set that up.

References

Edited by Andrew Jung

Merge request reports