Audit Events for agents and flows

About

We want to capture GitLab Audit Events when for the following changes:

• Item updated
• Item created
• Item made public
• Item made private
• Item draft version created
• Item version released
• Item deleted
• Item added to project
• Item removed from project

Note some of the above "made public", "version released" will happen at the same time as "created" or "updated" - this means we create multiple audit events in those situations.

For example, if someone creates a private agent with a released version, they would generate 2 events:

• "Created private AI agent"
• "Released version 1.0.0 of AI agent"

If someone updated an agent, switching it from private to public, with a new released version, they would generate 3 events:

• "Updated AI agent"
• "Made AI agent public"
• "Released version 1.1.0 of AI agent"

This will allow customers to have an audit trail of these events.

See:

• Customer docs https://docs.gitlab.com/user/compliance/audit_events/
• Developer docs https://docs.gitlab.com/development/audit_event_guide/

Proposal

Define audit events (see development docs).

Events will be scoped by the project and have the target as the item.

Update documentation https://docs.gitlab.com/user/compliance/audit_event_types/.

Agents

Event names:

• Name: create_ai_catalog_agent
  • Messages:
    • One of:
      • "Created a new public AI agent with tools [tool1, tool2]"
      • "Created a new private AI agent with tools: [tool1, tool2]"
    • One of:
      • "Created new draft version 1.0.0 of AI agent" (when version is in draft)
      • "Released version 1.0.0 of AI agent" (when version is being released)
• Name: update_ai_catalog_agent
  • Messages - one or more of:
    • "Updated AI agent: <description of changes>" (always generated)
      • Description of changes can be:
        • "Added tools: [tool1, tool2]" (when new tools added)
        • "Removed tools: [tool1, tool2]" (when new tools removed)
        • "Changed system prompt" (when system prompt updated)
    • "Made AI agent public" (when changes include private -> public)
    • "Made AI agent private" (when changes include public -> private)
    • "Created new draft version n.n.n of AI agent" (when latest version is new record and in draft)
    • "Released version n.n.n of AI agent" (when latest version changes from draft -> released, regardless of new record or old record)
• delete_ai_catalog_agent
  • Messages:
    • "Deleted AI agent" (always generated)
• enable_ai_catalog_agent
  • Messages:
    • "Added AI agent to project/group" (always generated)
• disable_ai_catalog_agent
  • Messages:
    • "Removed AI agent from project/group" (always generated)

Examples

Creating public agent with draft version:

audit_context = {
  name: 'create_ai_catalog_agent',
  author: current_user,
  scope: project,
  target: agent,
  message: 'Created a new public AI agent with tools [ci_linter]'
}

Gitlab::Audit::Auditor.audit(audit_context)

audit_context = {
  name: 'create_ai_catalog_agent',
  author: current_user,
  scope: project,
  target: agent,
  message: 'Created new draft version 1.0.0 of AI agent'
}

Gitlab::Audit::Auditor.audit(audit_context)

Updating agent to be public, and changing the tools:

audit_context = {
  name: 'update_ai_catalog_agent',
  author: current_user,
  scope: project,
  target: agent,
  message: 'Updated AI agent: Added tools [run_git_command], removed tools [ci_linter]'
}

Gitlab::Audit::Auditor.audit(audit_context)

audit_context = {
  name: 'update_ai_catalog_agent',
  author: current_user,
  scope: project,
  target: agent,
  message: 'Made AI agent public'
}

Gitlab::Audit::Auditor.audit(audit_context)

Enabling agent:

audit_context = {
  name: 'enable_ai_catalog_agent',
  author: current_user,
  scope: project,
  target: agent,
  message: 'Added AI agent to project'
}

Gitlab::Audit::Auditor.audit(audit_context)

Flows

• create_ai_catalog_flow
• update_ai_catalog_flow
• delete_ai_catalog_flow
• enable_ai_catalog_flow
• disable_ai_catalog_flow

Identical logic to agents. We will need to parse the YAML to collect the tool and prompt changes for AgentComponent components within the flow, see: #566901 (comment 2878260005).

Third Party Flows

• create_ai_catalog_third_party_flow
• update_ai_catalog_third_party_flow
• delete_ai_catalog_third_party_flow
• enable_ai_catalog_third_party_flow
• disable_ai_catalog_third_party_flow

Identical logic to agents (excluding tool changes as 3P flows do not use tools as such). See #566901 (comment 2886802787)