Duo Workflow Usage data

Problem to solve

To track product usage data for duo workflow.

Workflow execution for IDE https://handbook.gitlab.com/handbook/engineering/architecture/design-documents/duo_workflow/#with-local-ide-execution

Metrics to capture

For MVP

• workflow_start
• workflow_complete

Future scope

Workflow metrics

The goal of workflow metrics is to provide us information on the workflow in a way that will enable us to create dashboards that will show us 360 view on the activity

• Number of completed workflows
• Steps / Tasks per workflow
• Workflow duration
• Workflow token consumption
  • Prompt
  • Response
• Workflow errors
• Workflow interventions
  • Workflow stopped
  • Workflow plan edited by the user

User metrics

The key is to have user metrics set up so we could understand the relationship between users and workflow

• User activity - Active user
  • in a way that we could use to visualize DAU, WAU, MAU and retention metrics
• Workflows per user
• Workflow duration per user

Implementation Plan

• Add instrumentation events as per usage/scope
• Use track_event endpoint in monolith to send events.