AI Impact: [FE] Add Code Suggestions Acceptance Rate tile

Release note

AI Impact analytics - new insights into Code Suggestions: Acceptance Rate

Today, we’re excited to introduce a key metric that highlights the effectiveness of Duo Code Suggestions: Code Suggestions: Acceptance Rate %.

The Code Suggestions: Acceptance Rate % metric indicates how often the suggestions made by GitLab Duo are utilized by developers. This metric is a direct reflection of the effectiveness of GitLab Duo's suggestions and the level of trust that contributors have in its capabilities.

In addition to the new metrics, we introduce a new Overview tile visualization that provides a clear and concise summary of the metrics, allowing users to quickly assess the current state of the AI features.

Problem

Engineering leadership (Director/VP/CTO) need to analyze the effectiveness of the Code Suggestion functionality.

Related to these Customer feedback.

Proposal

Once the backend work on AI Impact: Expose Code Suggestions Acceptance R... (#456088 - closed) • Pavel Shutsin • 17.2 will be done, visualize the Code Suggestions: Acceptance Rate % in a new tile:

  1. UI Spec:
    • Tile title - Code Suggestions: Acceptance Rate %
    • ℹ️ 16,245 out of 23,890 code suggestions were accepted in the last 30 days.
    • See docs - https://docs.gitlab.com/ee/user/project/repository/code_suggestions/index.html#use-code-suggestions
  2. Objective: This metric indicates how often the suggestions made by the Gitlab are utilized by the developers, reflecting the effectiveness of Duo and the contributors' trust in its suggestions.
  3. Definition: The proportion of code suggestions provided by Gitlab Duo that are accepted by code contributors within the last 30 days.
  4. Calculation Formula: codeSuggestionsAcceptedCount / codeSuggestionsShownCount = Code Suggestions: Acceptance Rate %
  5. Aggregation Method: Average of the last 30 days
  • documentation need to be updated with this feedback.
Edited Aug 01, 2024 by Haim Snir
Assignee Loading
Time tracking Loading