Self-Service Model Selection Usage Reporting and Auditing

Problem Statement

Enterprise customers using GitLab Duo Model Selection through the cloud connector AI gateway currently have no visibility into their AI model usage patterns, routing decisions, or audit trails. This creates barriers for compliance, security oversight, and operational validation that these customers require.

Customer Requirements:

• Validation that their pinned models via Model Selection are being honored
• Self-service access to usage logs without requiring support tickets
• Audit trails for compliance and governance requirements
• RBAC for usage log access
• Privacy-compliant logging that balances transparency with data protection

Technical Considerations:

• Logs stored in normal log storage (currently ~7 days retention UPDATE: logging retention increased to 30 days for https://log.gprd.gitlab.net/)
• Logs are not currently available in Clickhouse / Snowplow

Proposal

Based on the technical approach discussion, this issue proposes implementing a log-based self-service reporting system that provides customers with downloadable usage reports through the GitLab UI.

Phase 1:

• Enhanced Logging: Ensure Model Selection logs contain the right information for customer reporting:
  • unique request identifier
  • Model routing (which model was selected/used)
  • Model routing decision (i.e. pinned at namespace, end-user-selected)
  • Namespace-level usage patterns per Duo feature/agent
  • Request metadata (timestamp, user, model selection logic applied, request source (ie IDE, UI), associated Duo feature)

Phase 2:

• Self-Service Report Generation:
  • Customers can request usage reports through the GitLab UI
  • "Generate Report" button triggers a background job
  • Job fetches logs from storage, filters by namespace
  • Returns downloadable file or UI-accessible link
  • Report format = CSV/JSON format containing:
    • Transaction-level details (each AI request/turn)
    • Model routing
    • Model routing decision (i.e. pinned at namespace, end-user-selected)
    • Aggregated usage summaries

Phase 3:

• Automated Report Provisioning:
  • Optional: Auto-generate reports every x days for active namespaces
  • Store reports as downloadable files in GitLab's filesystem
  • Provide UI notifications when new reports are available

Definition of Done

• Namespace owners who have pinned models via Model Selection can validate that their model request has been honored
• Namespace owners can trigger generation of a report in the GitLab UI, to include date range selection
  • Navigate to Namespace > Settings > GitLab Duo > "Generate Report"
  • Configure date and Click "Generate Usage Report" for desired date range
  • Background job processes request
  • User receives notification when report is ready
  • User directed to a link or downloadable report containing details
• Namespace owners can access and download reports
• Namespace owners can set retention periods for generated reports
• Documentation updates to help namespace owners understand and trigger reports