Add Snowplow-compatible Duo events collector endpoint
What does this MR do and why?
Adds a Snowplow-compatible collector endpoint so a self-hosted AI gateway on an air-gapped instance has a local place to send Duo (DAP) billing events:
POST /api/v4/ai/events_collector/com.snowplowanalytics.snowplow/tp2The endpoint is only reachable when the existing instance-level local_billing_persistence feature flag is enabled (the same flag that turns on local persistence in EE::Gitlab::BillingEvents::Client); otherwise it returns 404.
Authentication: the request must carry a personal access token that belongs to a service account and has the ai_features scope (api is also accepted, as on every endpoint). No token returns 401; a token with another scope, or one belonging to a human user, returns 403. The AI gateway sends the token through AIGW_BILLING_EVENT__API_KEY (gitlab-org/modelops/applied-ml/code-suggestions/ai-assist!7288 (merged)). Granular (fine-grained) token authorization is deferred with route_setting :authorization, todo: until the dedicated fine-grained scope exists.
The endpoint accepts the Snowplow tracker protocol payload the AI gateway's BillingEventsClient emits. Ai::EventsCollector::IngestService decodes the billable_usage context and forwards it to Gitlab::BillingEvents::Client.track_billing_event, which already routes events on offline-licensed instances to Utilization::BillableUsage::RecordAggregateService.
Still to do in follow-up iterations (not in this MR):
- Create the fine-grained global access scope for this endpoint and replace the
todo:route setting with it. - Add API docs plus admin setup docs (service account and token creation).
- Consider moving ingestion behind a Sidekiq job if batch sizes grow (
RecordAggregateServicenotes DAP volume would need batching).
References
Related to https://gitlab.com/gitlab-org/gitlab/-/work_items/631688
AI gateway counterpart: gitlab-org/modelops/applied-ml/code-suggestions/ai-assist!7288 (merged)
Screenshots or screen recordings
Not applicable, API-only change with no UI.
How to set up and validate locally
-
Enable billing event tracking and local persistence:
Feature.enable(:billing_event_tracking) Feature.enable(:local_billing_persistence) -
Create a service account and a token with the
ai_featuresscope in the Rails console:sa = Users::ServiceAccounts::CreateService.new(User.admins.first, { organization_id: Organizations::Organization.first.id }).execute.payload[:user] token = PersonalAccessToken.create!(user: sa, name: 'aigw-collector', scopes: %w[ai_features], expires_at: 30.days.from_now, organization: sa.organization) puts token.token -
Send a Snowplow tracker protocol payload with the token. Replace
NAMESPACE_IDwith an existing group ID andTOKENwith the value printed above:CX=$(printf '{"schema":"iglu:com.snowplowanalytics.snowplow/contexts/jsonschema/1-0-1","data":[{"schema":"iglu:com.gitlab/billable_usage/jsonschema/1-0-3","data":{"event_id":"%s","event_type":"duo_agent_platform_workflow_completion","unit_of_measure":"tokens","quantity":5328,"timestamp":"2026-10-02T10:00:00","namespace_id":NAMESPACE_ID,"metadata":{"feature_qualified_name":"duo_agent_platform"}}}]}' "$(uuidgen)" | base64 | tr '+/' '-_' | tr -d '=\n') curl --request POST \ --url "http://gdk.test:3000/api/v4/ai/events_collector/com.snowplowanalytics.snowplow/tp2" \ --header "Authorization: Bearer TOKEN" \ --header "Content-Type: application/json" \ --data "{\"schema\":\"iglu:com.snowplowanalytics.snowplow/payload_data/jsonschema/1-0-4\",\"data\":[{\"e\":\"se\",\"se_ca\":\"DuoWorkflowService\",\"se_ac\":\"duo_agent_platform_workflow_completion\",\"aid\":\"gitlab_ai_gateway\",\"tv\":\"py-1.0.0\",\"p\":\"srv\",\"cx\":\"$CX\"}]}" -
Expect
HTTP 200with body{"status":"ok"}and aBillingEvents:entry inlog/application_json.log. On an offline-licensed instance,Utilization::BillableUsage::DailyAggregate.lastshows the aggregated row. -
Repeat without the
Authorizationheader; expectHTTP 401. Repeat with a token of your own (human) user; expectHTTP 403withToken must belong to a service account. -
Run
Feature.disable(:local_billing_persistence)and repeat the request; expectHTTP 404.
Validate locally with the AIGW
Checkout to this MR follow the instruction on how to validate it locally and trigger a new DAP chat.
pipeline:skip-router-sync applied
The new were added routes in the cells https router repo gitlab-org/cells/http-router!1417 (merged) other routes are outputting the error.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.