Separate AI agent governance from the DAP external agents setting

Jean van der Waltrequested to merge
633673-ai-agent-governance-setting into master

What does this MR do and why?

GitLab AI Agent Governance (GAIG) records sessions from external AI coding agents (Claude Code, OpenCode, Codex, Cursor) that glab uploads through the endpoints under /api/v4/projects/:id/ai_agent/ (identities, sessions, audit_events).

Until now, those endpoints used the same project policy rule as External Agents in the Duo Agent Platform (DAP), so turning off the top-level group's duo_external_agents_enabled also silently stopped GAIG capture.

This MR gates GAIG on the new AI Governance setting, ai_governance_enabled, instead. That setting was added by !262303 (merged). It is opt-in (off by default) and cascades from the instance to groups and projects, like ai_audit_events_storage_enabled.

An earlier version of this MR added a separate ai_agent_governance_enabled setting on ai_settings and namespace_ai_settings. That is dropped: the umbrella setting already separates governance from DAP, and one switch is simpler for admins than two that cascade differently. A separate toggle for external agents can still be added later under the umbrella if it is needed.

What changes:

  • Project policy. create_ai_agent_identity, create_ai_agent_session, create_ai_agent_audit_event, update_ai_agent_session and read_ai_agent_session are prevented unless the project's ai_governance_enabled is true. The DAP rule now only prevents the two third-party flow permissions, as before.
  • Personal namespaces. On GitLab.com, projects in a personal namespace stay off, because only groups can turn AI Governance on. On GitLab Self-Managed and GitLab Dedicated they follow the project value.
  • Groups API. PUT /api/v4/groups/:id accepts ai_governance_enabled and lock_ai_governance_enabled, the same way as ai_audit_events_storage_enabled. !262303 (merged) has no API or UI, so without this there is no way to turn GAIG on. Only group Owners can change it, the change is audited, and it is applied to subgroups and projects by the existing cascade worker.
  • User docs. The Externally Connected Agents page explains how to turn on AI Governance and replaces the old Allow external agents prerequisite.

Default false makes GAIG opt-in. Groups and instances using GAIG today stop recording when this deploys, until AI Governance is turned on. The backfill is tracked in https://gitlab.com/gitlab-org/gitlab/-/work_items/633222.

Not in this MR:

  • An instance-wide toggle for administrators. The application setting has no API or admin UI yet. Administrators can turn it on for each group through the groups API.
  • UI. That belongs to the AI Governance settings page in https://gitlab.com/gitlab-org/gitlab/-/work_items/632551.
  • A decision on licensing. The GAIG endpoints still also require the ai_catalog licensed feature.

References

How to set up and validate locally

  1. Check out the branch.

  2. As a developer of a project in a group, call POST /api/v4/projects/:id/ai_agent/identities with agent_type=claude-code and a 64-character hex machine_fingerprint. It returns 403.

  3. As a group Owner, turn on AI Governance:

    curl --request PUT --header "PRIVATE-TOKEN: <token>" \
  --data "ai_governance_enabled=true" "http://gdk.test:3000/api/v4/groups/<group_id>"

  4. Let Sidekiq run Namespaces::CascadeDuoSettingsWorker, then repeat the identity call. It returns 201.

  5. Turn off Allow external agents for the top-level group. The identity call still returns 201.

Tests

  • Policy specs: on and off for the project, an instance lock, GitLab.com, and personal namespaces on GitLab.com and self-managed.
  • The GAIG request specs for every endpoint, including the DAP decoupling and instance-lock cases. Each turns AI Governance on for the project in its top-level before.
  • Groups API specs for both new params.
  • ee/spec/lib/authz/role_definitions_spec.rb turns AI Governance on in open_policy_gates.
  • The identities, sessions and audit events request specs use :clean_gitlab_redis_rate_limiting, because their examples share one rate-limit counter in Redis.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Jean van der Walt

Merge request reports