Expand on OMRT KPI
Idea
Code Review Effectiveness Analysis (productivity metric)
Objective
Evaluate the effectiveness of code reviews and their impact on code quality, collaboration, and productivity.
Context
Code reviews are a crucial part of the development process, where team members can provide feedback on each other’s code to ensure code quality, maintainability, and adherence to coding standards. We measure Open MR Review Time (OMRT) - time it takes from when an MR is assigned for a review to when it's merged - but sometimes it is difficult to explain the fluctuations.
- Are there bottlenecks in the review process?
- Are there correlations between review characteristics and code quality?
Deep dive dashboard
https://app.periscopedata.com/app/gitlab/1156768/OMRT-deep-dive
Findings
- No correlation between OMRT and reviewers/files changed/added lines
- A lot of open MRs have undefined group MRs
- Bugs found by the community are heavily influencing OMRT. Maintenance seems pretty stable this year and features have a notable increase over time.
- Sankey diagram of open MRs and recency of activity #266 (comment 1655230251)
Data
Data collection
- MR creation date
- Number of reviewers
- Number of review comments (do we have reviewers who were not assigned as a reviewer?)
- review durations
- string search on MRs
- reopened/linked MRs or issues
- subsequent changes made on review feedback
Sub-metrics
PI name | Calculation | Analysis |
---|---|---|
Review duration | MR creation to approval | Identify outliers, such as merge requests with unusually long review durations, and investigate the reasons behind them. |
Review Iterations | # of iterations before approval | Is # of iterations correlated to time to review? Analyze specific merge requests that required a high number of iterations to identify potential reasons, such as unclear requirements or frequent scope changes. |
Reviewer Participation | # of reviewers | The distribution of review workload among team members, and the average time taken to provide feedback. |
Code Quality Improvement | # of bugs/issues before or after code reviews. Reopened or linked MR | Investigate the impact of code reviews on code quality. |
Review Feedback Analysis | Review comments and feedback string search | Categorize review comments into different types, such as bug reports, suggestions for improvement, or comments related to code style. |
Developer Collaboration | Response times to review comments, specific keywords/phrases that indicate conflict/disagreement | Identify potential areas for enhancing collaboration and communication within the team. |
Correlation Analysis | Investigate if higher reviewer participation correlates with faster merge request completion times. |
Edited by Lily Mai