Keep only tool name in AI governance tool-call dedup

What this does

The AI Governance dashboard counts agent tool calls from ClickHouse. Before counting, it removes duplicate events. That step kept each event's full details payload in memory, even though only the tool name is used. This MR keeps just the tool name, so memory use no longer grows with the size of the payloads. The counts stay the same.

Why

details holds prompts and tool arguments and can be several KB per event. One dashboard load runs this step five times, and the default 7-day view reads 65 days of events. On a group with a lot of agent activity, the queries could hit the ClickHouse memory limit, and the tool call cards would fail to load.

Before and after

The dedup step, before:

SELECT traversal_path, workflow_id, created_at, id, any(details) AS details
FROM ai_audit_events
...
GROUP BY traversal_path, workflow_id, created_at, id

After:

SELECT traversal_path, workflow_id, created_at, id, any(<tool name expr>) AS tool_name
FROM ai_audit_events
...
GROUP BY traversal_path, workflow_id, created_at, id

Peak memory, measured with clickhouse local 26.9.2 on synthetic events with about 4 KB of details, one 30-day window:

Events Before After
200,000 (~770 MiB of details) 1.25 to 1.32 GB ~240 MB
400,000 (~1.5 GiB of details) 2.21 GB ~200 MB

The counts were identical, and query time barely changed.

No visual change

The Tool calls tile, the Top tools card and the Tool calls by risk card show the same numbers. Only the memory the queries use changes. Captured on local data, Last 30 days, same group and window, cropped to each card:

Card Before After
Tool calls tile tool-calls-tile-before tool-calls-tile-after
Top tools used by agents top-tools-card-before top-tools-card-after

The images are identical. The Tool calls by risk card reads the same query as the Top tools card, and run both ways on the same data it returns identical rows.

Why the counts don't change

Duplicate rows are identical. Taking the tool name from any one of them gives the same answer as keeping the whole payload and reading the name later. The existing specs already count a re-ingested duplicate once.

Changes

  • ClickHouseToolCallsService (Top tools and Tool calls by risk cards): the dedup keeps the tool name instead of details.
  • ClickHouseMetricsService (Tool calls tile: total, trend and 30-day baseline): the same change, in all three places that read the dedup.
  • A spec check in both that the dedup no longer selects the full details payload.

Out of scope

  • A stored tool_name column on ai_audit_events would help other readers too, but it needs a ClickHouse migration and a backfill.
  • The risk card's by-agent query in !261725 (merged) gets the same change once that MR merges.

Related to #635324.

Edited by Dheeraj Joshi

Merge request reports

Loading
Loading