Add the tool calls card, ClickHouse reader and GraphQL field to the AI Governance dashboard
What does this MR do and why?
Adds the "Top tools used by agents" card and a "Total number of tool calls" summary tile to the AI Governance dashboard, with the ClickHouse reader and GraphQL fields behind them. Both sit behind the ai_governance_tool_calls flag (wip, off by default).
This MR now includes the summary tile from Add AI tool calls summary tile to the AI Govern... (!256856 - merged) • Dheeraj Joshi • 19.5, which was reviewed separately and merged into this branch.
Card:
Ai::Governance::ClickHouseToolCallsService: top 10 tools by call count for a group or project, timeframe and agent class, fromai_audit_eventsin ClickHouse.Ai::Governance::ToolCallsService: uses ClickHouse whenGitlab::ClickHouse.enabled_for_analytics?is on and returns an error otherwise. It owns the sharedEVENTSandTOP_Nconstants.aiGovernanceToolCallsonGroupandProject(experiment): tool invocations by tool name for a timeframe and agent class, plus anapproximateflag. Nullable, gated behindai_governance_dashboardandai_governance_tool_calls.- The card: one row per tool with its call count for the dashboard's date range and agent class, a note when counts are approximate, and a link to the Tool management tab. When the field is null (no ClickHouse) it shows its empty state.
Tile:
- A
toolCallsKPI onaiGovernanceMetricswithcount,previousCount,trendandcumulativeTrend. The running total is seeded from the 30 days before the window, the same as sessions. - It counts every
ai_tool_invokedinvocation once, with the same dedup and tool-name rule as the card, so the tile total and the card agree. - Agent class is resolved through the workflow, the same way the card does it.
- ClickHouse only. Without ClickHouse the field is null and the tile shows a dash.
The PostgreSQL reader and KPI for instances without ClickHouse follow in Add tool-call tally scopes and PostgreSQL tool ... (!256523 - closed) • Dheeraj Joshi • 19.6 and plug into the same services.
Example query
query {
group(fullPath: "gitlab-org") {
aiGovernanceToolCalls(timeframe: LAST_7_DAYS, agentClass: ALL) {
approximate
nodes {
toolName
count
}
}
}
}Screenshots
| Tiles, last 30 days |
|---|
![]() |
| Card | Dark mode | Without ClickHouse |
|---|---|---|
![]() |
![]() |
![]() |
| Tile, dark mode | Tiles without ClickHouse |
|---|---|
![]() |
![]() |
How to validate locally
Needs ClickHouse enabled for analytics (Gitlab::ClickHouse.enabled_for_analytics? true) and audit events flowing into it.
- Enable the flags for a group:
Feature.enable(:ai_governance_dashboard, group)andFeature.enable(:ai_governance_tool_calls, group). - Run a few Duo agent sessions in a project of that group so
ai_tool_invokedaudit events land in ClickHouse, or insert rows intoai_audit_eventsandsiphon_duo_workflows_workflowsasee/spec/services/ai/governance/click_house_tool_calls_service_spec.rbdoes. Add one row older than 30 days and one with an emptytool_name: neither should show. - Open the group's Settings > GitLab Duo > Governance, Dashboard tab. The "Top tools used by agents" card lists the tools by call count.
- Switch the date range and the agent class filter. The card follows both.
- Click View tool management. It opens the Tool management tab.
- Check the "Total number of tool calls" tile, its delta badge and its trend. Confirm it follows the date range and the agent class filter.
- Query the API in GraphiQL with the example query above. The counts match the card.
- Query
aiGovernanceMetrics { toolCalls { count previousCount trend } }and compare with the tile. - Disable ClickHouse for analytics. The card field returns null and the card shows its empty state. The tile shows a dash.
ClickHouse review
ai_audit_events is a ReplacingMergeTree partitioned by month, ORDER BY (traversal_path, workflow_id, created_at, id). The query filters on traversal_path prefix and created_at, both in the
sort key, and deduplicates on the full sort key before counting because ReplacingMergeTree collapses
re-ingested rows only on merge. agent_type lives on siphon_duo_workflows_workflows
(ORDER BY (traversal_path, created_at, id)), so the agent-class filter is a semi-join on workflow_id;
it has no lower time bound because a session can start long before the events it emits into the window.
The tile's KPI queries (totals over the current and previous window, per-bucket trend, and a baseline for the running total) reuse the same dedup subquery and agent-class filter as the card's query, in ClickHouseMetricsService.
I don't have access to production ClickHouse to attach a plan. The spec runs the query against a real
ClickHouse server; if a reviewer can run EXPLAIN indexes=1 on gitlab-org, I'd welcome it.
Query, group scope, :internal_dap (:all drops the workflow_id IN block)
SELECT
JSONExtractString(details, 'tool_name') AS tool_name,
count() AS calls
FROM (
SELECT traversal_path, workflow_id, created_at, id, any(details) AS details
FROM ai_audit_events
WHERE startsWith(traversal_path, {traversal_path:String})
AND created_at >= {from:DateTime64(6, 'UTC')}
AND created_at < {to:DateTime64(6, 'UTC')}
AND event_name IN {events:Array(String)}
AND workflow_id IN (
SELECT id FROM (
SELECT id,
argMax(agent_type, _siphon_replicated_at) AS agent_type,
argMax(_siphon_deleted, _siphon_replicated_at) AS deleted
FROM siphon_duo_workflows_workflows
WHERE startsWith(traversal_path, {traversal_path:String})
AND created_at < {to:DateTime64(6, 'UTC')}
GROUP BY traversal_path, created_at, id
)
WHERE deleted = false
AND agent_type IS NULL
)
GROUP BY traversal_path, workflow_id, created_at, id
)
WHERE tool_name != ''
GROUP BY tool_name
ORDER BY calls DESC, tool_name ASC
LIMIT {limit:UInt64}Placeholders: traversal_path is the namespace's traversal_path(with_organization: true), for example
1/9970/; events is ['ai_tool_invoked']; from/to are the dashboard window, upper bound
exclusive; limit is 10. :external uses agent_type IS NOT NULL.
Specs
ee/spec/services/ai/governance/click_house_tool_calls_service_spec.rb(runs against ClickHouse)ee/spec/services/ai/governance/tool_calls_service_spec.rbee/spec/services/ai/governance/click_house_metrics_service_spec.rbee/spec/requests/api/graphql/ai/governance/tool_calls_spec.rb(runs against ClickHouse)ee/spec/requests/api/graphql/ai/governance/metrics_tool_calls_spec.rbee/spec/requests/api/graphql/ai/governance/metrics_spec.rbee/spec/frontend/ai/governance/components/dashboard/cards/tool_calls_card_spec.jsee/spec/frontend/ai/governance/components/dashboard/dashboard_app_spec.js
Follow-ups
Agent class is resolved through the workflow today. When ai_governance_sessions lands (#630095),
this moves to the session's source.





