UX: [VSD] "AI Impact view" MVC - visualize the relationship between "AI usage" trends and the SDLC performance.

Problem

Engineering leadership (Director/VP/CTO) need to demonstrate the ROI from the investments in AI features. To quantify the impact of adding AI and to demonstrate the ROI, they need to:

  1. Understand which metrics improved as a result of this investments in AI.
  2. To compare the performance of teams that are using AI against teams that are not using AI.

Personas

  • Dakota (Application Development Director)
  • Erin (Application Development Executive)
  • Delaney (Development Team Lead)

MVC Proposal

Visualize the relationship between "AI usage" trends and the SDLC trends to demonstrate the ROI of AI adoption.

  1. For the MVC, we assume that we can use "Code Suggestions Usage" as the leading indication of "AI usage" in a group.
  2. Using the Code Suggestions usage data to count the Code Suggestions monthly unique users for a given group.
  3. Using the groupoptimize ClickHouse-based contributors table to count Total monthly unique users for a given group.
  4. Calculated the monthly "AI Usage rate" = (Code Suggestions monthly unique users / Total monthly unique users) for a given group.
  5. Adding new comparison table to compare the new "AI Usage rate" (Independent variable) vs the SDLC metrcis (Dependent variables).
Preview
AI_impact_-_MVC_code_suggestion_focused
⚠ This preview maybe up-of-date. See Design management for latest mockups
Old proposal
Metrics for "Group ABC" Jan Feb Mar Apr May Jun Change (%)
Cycle time (Days) 10 9.8 9.6 9.4 9.2 9 -10.0%
Lead time 30 29 28 27 26 25 -16.67%
Deployment Frequency (per day) 150 152 154 156 158 160 6.67%
Change failure rate 0.5 0.49 0.48 0.47 0.46 0.45 -10.00%
Critical vulnerabilities 100 99 98 97 96 95 -5.00%
AI Usage rate - Unique/total 0.1 0.2 0.3 0.4 0.5 0.6 500.0%

Next iterations (not in priority order)

  1. Adding "AI features" adoption view/table.
  2. Adding "AI Impact" comparison between AI groups vs Non-AI groups.
  3. Adding algorithm to highlight significant "AI Impact" correlations for the entire organization/namespace.
  4. Adding the rest of GL AI features.

Solution validation

Resources

Edited by Libor Vanc