Add a CATALOG agent class to AI governance metrics (!247948) · Merge requests · GitLab.org / GitLab

What does this MR do and why?

Adds a CATALOG value to the AiGovernanceAgentClass enum so AI Catalog agents stop being reported as internal Duo Agent Platform agents.

Closes #607564. Stacked on !246384 (merged), which introduced the enum. Retarget to master once that merges.

The problem

INTERNAL_DAP was defined as agent_type IS NULL, a catch-all rather than a positive predicate, so it absorbed anything no external producer wrote. AI Catalog agents run inside DAP without an agent_type, so they landed there.

Measured on gitlab-production-main: about 88,000 sessions per 30 days, roughly 15% of all sessions, carry ai_catalog_item_version_id and are currently counted as internal (plan). No source describes them as internal. doc/user/duo_agent_platform/agents/external.md has called them External agents since 18.3, Ai::FlowTriggers::RunService errors with "not permitted to execute external agent", and the GAIG design doc (handbook MR 20111) puts them outside its scope as "a product registry".

The change

Class	Predicate
`ALL` (default)	no predicate
`INTERNAL_DAP`	`agent_type IS NULL AND ai_catalog_item_version_id IS NULL`
`CATALOG`	`agent_type IS NULL AND ai_catalog_item_version_id IS NOT NULL`
`EXTERNAL`	`agent_type IS NOT NULL`

INTERNAL_DAP gains the second condition so the three classes still partition the row set and ALL remains exactly their sum, which is asserted in a spec on both backends.

AGENT_INSTANCE_KEY gains ai_catalog_item_version_id, so two catalog agents run by the same user in the same project count as two instances rather than one. Internal and external rows have the column NULL, so their counts are unchanged.

Why this column

ai_catalog_item_version_id is on duo_workflows_workflows and is siphoned to ClickHouse, so the same predicate works on both read paths with no migration and nothing needed from another team.

Two alternatives were rejected on evidence. Joining ai_catalog_items for item_type would isolate third_party_flow agents specifically, but no catalog table is siphoned, so it only works on PostgreSQL. service_account_id is set on about 339,000 sessions per 30 days, roughly 57% of all sessions (plan), far broader than catalog runs.

Known limitations

CATALOG means "provisioned from the AI Catalog", which includes foundational agents, user-created agents and flows, not only third-party external agents. Subdividing needs the item_type join described above. Naming it for provenance also avoids taking a position on the unresolved "external agent" terminology, tracked in #606071.

The key uses the item version rather than the item, because the workflow row only carries the version. A version bump therefore reads as a new instance.

Ai::FlowTriggers::RunService writes no catalog column at all, so trigger-driven runs are still classified as internal. That is a gap in the flow-trigger path rather than in the metrics layer, recorded in #607564.

Database notes

No migration and no new index. The predicate adds a condition on an already-selected row set and the aggregate gains one column, from a five-column key to a six-column one. Plans to be re-captured against this query shape before merge; the plans on !246384 (merged) cover the previous shape.

How to validate

user = User.find_by_username('root')
Feature.enable(:ai_governance_dashboard)

query = <<~GQL
  query {
    group(fullPath: "your-group") {
      all:      aiGovernanceMetrics(agentClass: ALL)           { sessions { count } agents { count } }
      internal: aiGovernanceMetrics(agentClass: INTERNAL_DAP)  { sessions { count } agents { count } }
      catalog:  aiGovernanceMetrics(agentClass: CATALOG)       { sessions { count } agents { count } }
      external: aiGovernanceMetrics(agentClass: EXTERNAL)      { sessions { count } agents { count } }
    }
  }
GQL
puts GitlabSchema.execute(query, context: { current_user: user }).to_h

Validated locally against seeded data containing two catalog runs that share a user and project and differ only in ai_catalog_item_version_id:

agentClass	sessions	agents
`ALL`	12	10
`INTERNAL_DAP`	4	3
`CATALOG`	2	2
`EXTERNAL`	6	5

Identical on the PostgreSQL and ClickHouse paths, and 10 == 3 + 2 + 5 holds on both. Reverting either the key addition or the INTERNAL_DAP condition fails seven examples across the two service specs, including the partition invariant.

References

Work item: #607564
Parent: !246384 (merged)
Taxonomy question: #606071

Add a CATALOG agent class to AI governance metrics