Skip to content

Tracking Instrumentation: Add LLM model metadata to gitlab_duo_workflow events

Summary

Add LLM model metadata to all Snowplow events where app_id = gitlab_duo_workflow. Fields already available in the Snowplow context that should be included consistently are: model_engine, model_name, and model_provider.

Background

Per request from leadership, we need to filter analytics and dashboards by the LLM model used. While newly instrumented token events include model metadata, many Duo Workflow Service events do not. Standardizing these fields across all gitlab_duo_workflow events will enable consistent analysis and Tableau dashboard filtering.

Related work: recent MR added token tracking with model metadata !3161 (merged).

Scope

  • Ensure the following fields are populated for every Snowplow event with app_id = gitlab_duo_workflow:
    • model_engine
    • model_name
    • model_provider
  • If any of these fields are unavailable for a given event type, document the gap and reasoning

Acceptance Criteria

  • All events emitted with app_id = gitlab_duo_workflow include model_engine, model_name, and model_provider in their Snowplow context.
  • Reviewed/approved by Analytics Instrumentation.

Collaboration

  • Loop in Analytics Instrumentation for implementation guidance and review (they are working on an AI event standardization effort: gitlab-org&19545)
  • Duo Agent Platform engineering for implementation

Verification

  • Use snowplow micro or snowplow inspector to verify
  • Loop in PDI counterpart and Analytics Instrumentation for review
Edited by Emma Fergen