Add agentClass filter to Duo Workflow session artifacts

What

Adds an agentClass argument to duoWorkflowSessionArtifacts (group and project), so AI Governance session data can be filtered by agent class: All, DAP (INTERNAL_DAP, agent_type IS NULL), or externally-connected (EXTERNAL, agent_type IS NOT NULL).

Why

Backend half of the DAP/ECA dashboard filter (#618950). The metrics API already supports agentClass; this brings the Audit logs card and Audit events tab to parity.

How

  • agentClass is translated to a boolean agent_type_present in BaseSessionArtifactsResolver, so the GraphQL enum stops at the GraphQL boundary. ALL maps to nil and leaves the query unfiltered.
  • Both finders filter duo_workflow_session_artifacts.agent_type for null / not-null. Not ClickHouse-gated.

Database review

agent_type was denormalized onto duo_workflow_session_artifacts by !252345 (merged) (kept in sync by SessionArtifact.sync_from_workflow!), so this filter is a nullability check on the table's own column. SessionArtifact.with_agent_type_present is applied to the scope handed to the existing in-operator keyset pagination in PostgresqlFinder, which pushes the predicate inside the per-namespace LEFT JOIN LATERAL ... LIMIT 1. nil is a no-op (all), leaving the query unchanged.

An earlier revision of this MR reached agent_type through a workflow_id semi-join into duo_workflows_workflows, before !252345 (merged) landed. That is gone. Two things follow: the filter and the agentType field now read the same column and cannot disagree, and the predicate can early-limit through the existing index instead of materializing the whole namespace-scoped set ahead of the top-N LIMIT.

PostgreSQL query

The filter lands inside the lateral that the in-operator optimization already runs per namespace. Trimmed to the relevant fragment, for INTERNAL_DAP:

LEFT JOIN LATERAL (
  SELECT duo_workflow_session_artifacts.workflow_updated_at,
         duo_workflow_session_artifacts.workflow_id
  FROM duo_workflow_session_artifacts
  WHERE duo_workflow_session_artifacts.agent_type IS NULL
    AND duo_workflow_session_artifacts.namespace_id = array_cte.id
  ORDER BY duo_workflow_session_artifacts.workflow_updated_at DESC,
           duo_workflow_session_artifacts.workflow_id DESC
  LIMIT 1
) duo_workflow_session_artifacts ON TRUE

EXTERNAL is identical with agent_type IS NOT NULL. The same predicate appears in the recursive term's cursor lateral.

EXPLAIN ANALYZE (postgres.ai, gitlab-production-main)

Shareable plans: INTERNAL_DAP | EXTERNAL

Setup, so the numbers can be read fairly. The PG table is near-empty in production (ClickHouse is the primary store), and !252345 (merged) has merged but not yet deployed, so the clone was given the column with that migration's DDL (ADD COLUMN agent_type text + check_4d534409f3) and seeded with 50k artifacts under gitlab-org (9970): 45k agent_type IS NULL, 5k 'claude-code', one artifact per workflow, ANALYZE after. Both figures below are the warm second run.

execution planning buffers I/O reads
INTERNAL_DAP 36.0 ms 3.4 ms 30,815 hit 0
EXTERNAL 49.5 ms 5.4 ms 30,819 hit 0

Both stay under the 100 ms guideline with no I/O. What the plans show on the artifacts table:

  • The predicate rides the existing index in both laterals: Index Scan using index_duo_wf_session_artifacts_on_ns_wf_updated_wf_id, Filter: (agent_type IS [NOT] NULL). No sequential scan, no sort, no new index.
  • The paging steps early-limit. 20 loops at 0.008 ms (INTERNAL_DAP) and 0.010 ms (EXTERNAL), 60 and 64 buffer hits respectively. The previous semi-join revision produced a 45,000-row Nested Loop at 241,360 buffer hits for the same page.
  • EXTERNAL reports Rows Removed by Filter: 9 per paging step, which is the seeded 1-in-10 external density: the index walk skips ~9 internal rows to find each external one. INTERNAL_DAP skips none. See the caveat below.

Most of the remaining time is the pre-existing array_cte, not this filter: the namespaces traversal_ids bitmap heap scan over 10,553 namespaces costs 21.3 ms / 27.5 ms and 9,579 of the ~30,815 buffers. That is the group-hierarchy scope the finder already used before this MR. On a cold clone the same node takes ~8 s of first-touch block reads, which is a Database Lab artifact rather than a property of the query.

Indexes / selectivity

  • index_duo_wf_session_artifacts_on_ns_wf_updated_wf_id on (namespace_id, workflow_updated_at DESC, workflow_id DESC) serves the namespace scope and the ordering. agent_type is applied as a row filter while that index is walked in order, so the lateral stops at the first match instead of sorting the whole namespace set.
  • No index on agent_type is added. It is a low-cardinality NULL / NOT-NULL split that a plain btree indexes poorly, and it is not driving the scan.
  • Caveat: EXTERNAL in a namespace with few or no externally-connected sessions makes the lateral walk that namespace's index range before the LIMIT is satisfied. If that shows up as expensive on the plan, the fix is a partial index on (namespace_id, workflow_updated_at DESC, workflow_id DESC) WHERE agent_type IS NOT NULL.

ClickHouse

ClickHouseFinder#filter_by_agent_type_present adds an agent_type IS [NOT] NULL predicate to the outer query, after the existing dedup (one value per workflow via the PK GROUP BY / argMax). agent_type was already in COLUMNS on master because SessionArtifactType exposes agentType, so this adds no new projection and no new scan pattern.

Requesting a database review.

cc @andrew.jung for review, and to confirm the filter-gating placement (PG vs ClickHouse) is what you expect.

Edited by Dheeraj Joshi

Merge request reports

Loading
Loading