Duo RCA/Chat: Usage metrics show incorrect denominator on Duo Trends dashboard
Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.
Problem
On the Duo Trends dashboard (https://gitlab.com/groups/gitlab-org/-/analytics/dashboards/duo_and_sdlc_trends), the Duo RCA: Usage and the Duo Chat: Usage metrics show an incorrect denominator when hovering over the values month over month.
The denominator appears to be the same for month and there is no change month to month which is unexpected behavior. According to the metric definitions, Duo RCA: Usage should have its own specific denominator that reflects the appropriate basis for calculating RCA usage percentages.
Expected Behavior
- The Duo RCA: Usage metric should have its own appropriate denominator that reflects the actual basis for calculating RCA usage percentages
- The denominator should be different from other Duo metrics as each has distinct calculation methods
- Users should be able to properly interpret RCA usage statistics with the correct context
- Both Usage metrics should likely see changes in the denominators month over month, not static values over time
Current Impact
- Customer escalation due to confusing and incorrect metric calculations, leading to lack of trust in our data (Highmark)
- Users cannot properly interpret the Duo RCA usage statistics
- Dashboard provides misleading information about Duo RCA adoption rates
- Inconsistent metric definitions across the dashboard
Steps to Reproduce
- Navigate to https://gitlab.com/groups/gitlab-org/-/analytics/dashboards/duo_and_sdlc_trends
- Locate the "Duo RCA: Usage" metric
- Hover over the values to see the denominator
- Compare with other metrics (like Duo Chat: Usage) and notice they show the same denominator
- Verify that the denominator should be different based on the metric definitions
Additional Context
This is part of a customer escalation and needs attention to ensure accurate reporting of Duo RCA metrics. The issue affects the reliability of usage analytics for Duo features.
Example screenshots:
