Add agentClass filter to Duo Workflow session artifacts
What
Adds an agentClass argument to duoWorkflowSessionArtifacts (group and project), so AI Governance session data can be filtered by agent class: All, DAP (INTERNAL_DAP, agent_type IS NULL), or externally-connected (EXTERNAL, agent_type IS NOT NULL).
Why
Backend half of the DAP/ECA dashboard filter (#618950). The metrics API already supports agentClass; this brings the Audit logs card and Audit events tab to parity.
How
agentClassis translated to a booleanagent_type_presentinBaseSessionArtifactsResolver, so the GraphQL enum stops at the GraphQL boundary.ALLmaps toniland leaves the query unfiltered.- Both finders filter
duo_workflow_session_artifacts.agent_typefor null / not-null. Not ClickHouse-gated.
Database review
agent_type was denormalized onto duo_workflow_session_artifacts by !252345 (merged) (kept in sync by SessionArtifact.sync_from_workflow!), so this filter is a nullability check on the table's own column. SessionArtifact.with_agent_type_present is applied to the scope handed to the existing in-operator keyset pagination in PostgresqlFinder, which pushes the predicate inside the per-namespace LEFT JOIN LATERAL ... LIMIT 1. nil is a no-op (all), leaving the query unchanged.
An earlier revision of this MR reached agent_type through a workflow_id semi-join into duo_workflows_workflows, before !252345 (merged) landed. That is gone. Two things follow: the filter and the agentType field now read the same column and cannot disagree, and the predicate can early-limit through the existing index instead of materializing the whole namespace-scoped set ahead of the top-N LIMIT.
PostgreSQL query
The filter lands inside the lateral that the in-operator optimization already runs per namespace. Trimmed to the relevant fragment, for INTERNAL_DAP:
LEFT JOIN LATERAL (
SELECT duo_workflow_session_artifacts.workflow_updated_at,
duo_workflow_session_artifacts.workflow_id
FROM duo_workflow_session_artifacts
WHERE duo_workflow_session_artifacts.agent_type IS NULL
AND duo_workflow_session_artifacts.namespace_id = array_cte.id
ORDER BY duo_workflow_session_artifacts.workflow_updated_at DESC,
duo_workflow_session_artifacts.workflow_id DESC
LIMIT 1
) duo_workflow_session_artifacts ON TRUEEXTERNAL is identical with agent_type IS NOT NULL. The same predicate appears in the recursive term's cursor lateral.
EXPLAIN ANALYZE (postgres.ai, gitlab-production-main)
Shareable plans: INTERNAL_DAP | EXTERNAL
Setup, so the numbers can be read fairly. The PG table is near-empty in production (ClickHouse is the primary store), and !252345 (merged) has merged but not yet deployed, so the clone was given the column with that migration's DDL (ADD COLUMN agent_type text + check_4d534409f3) and seeded with 50k artifacts under gitlab-org (9970): 45k agent_type IS NULL, 5k 'claude-code', one artifact per workflow, ANALYZE after. Both figures below are the warm second run.
| execution | planning | buffers | I/O reads | |
|---|---|---|---|---|
INTERNAL_DAP |
36.0 ms | 3.4 ms | 30,815 hit | 0 |
EXTERNAL |
49.5 ms | 5.4 ms | 30,819 hit | 0 |
Both stay under the 100 ms guideline with no I/O. What the plans show on the artifacts table:
- The predicate rides the existing index in both laterals:
Index Scan using index_duo_wf_session_artifacts_on_ns_wf_updated_wf_id,Filter: (agent_type IS [NOT] NULL). No sequential scan, no sort, no new index. - The paging steps early-limit. 20 loops at 0.008 ms (
INTERNAL_DAP) and 0.010 ms (EXTERNAL), 60 and 64 buffer hits respectively. The previous semi-join revision produced a 45,000-row Nested Loop at 241,360 buffer hits for the same page. EXTERNALreportsRows Removed by Filter: 9per paging step, which is the seeded 1-in-10 external density: the index walk skips ~9 internal rows to find each external one.INTERNAL_DAPskips none. See the caveat below.
Most of the remaining time is the pre-existing array_cte, not this filter: the namespaces traversal_ids bitmap heap scan over 10,553 namespaces costs 21.3 ms / 27.5 ms and 9,579 of the ~30,815 buffers. That is the group-hierarchy scope the finder already used before this MR. On a cold clone the same node takes ~8 s of first-touch block reads, which is a Database Lab artifact rather than a property of the query.
Indexes / selectivity
index_duo_wf_session_artifacts_on_ns_wf_updated_wf_idon(namespace_id, workflow_updated_at DESC, workflow_id DESC)serves the namespace scope and the ordering.agent_typeis applied as a row filter while that index is walked in order, so the lateral stops at the first match instead of sorting the whole namespace set.- No index on
agent_typeis added. It is a low-cardinality NULL / NOT-NULL split that a plain btree indexes poorly, and it is not driving the scan. - Caveat:
EXTERNALin a namespace with few or no externally-connected sessions makes the lateral walk that namespace's index range before theLIMITis satisfied. If that shows up as expensive on the plan, the fix is a partial index on(namespace_id, workflow_updated_at DESC, workflow_id DESC) WHERE agent_type IS NOT NULL.
ClickHouse
ClickHouseFinder#filter_by_agent_type_present adds an agent_type IS [NOT] NULL predicate to the outer query, after the existing dedup (one value per workflow via the PK GROUP BY / argMax). agent_type was already in COLUMNS on master because SessionArtifactType exposes agentType, so this adds no new projection and no new scan pattern.
Requesting a database review.
cc @andrew.jung for review, and to confirm the filter-gating placement (PG vs ClickHouse) is what you expect.