Skip to content

Integrate Reviewer Recommender via the CI template for gitaly

Alexander Chueshev requested to merge ci-reviewer-recommender-v0.2.0 into master

Pre-requisites: Reviewer Recommender (aka UnReview) is an ML-based code reviewer recommendation system acquired by GitLab. By integrating a beta version of Reviewer Recommender into the Gitaly team, our Applied ML group would like to collect feedback to improve the quality of recommendations and help the team control the review workload.

Integration: This MR allows getting code reviewer recommendations when running the reviewers-recommender job included via the CI template. The reviewers-recommender job generates an artifact file with top-n recommendations, which can be used as a suggestion for assigning code reviewers. After implementing reviewer-recommender-bot v0.1.0, Reviewer-Recommender now puts recommendations into an MR note similar to Dangerbot. At each CI job run, Reviewer-Recommender searches for its previous comment within the first 10 notes. If the comment is found, then RR updates it. If not, then RR creates a new comment. Multiple RR comments have a very low probability of appearing in newly created MRs. In general, this logic is implemented to limit the number of heavy API GitLab calls and will be updated after unblocking https://gitlab.com/gitlab-org/modelops/applied-ml/review-recommender/gcp-postgres/-/issues/3.

Configuration:

  • The reviewers-recommender job doesn't change the existing review workflow.
  • Reviewer Recommender periodically collects the merge request history and trains the model.
  • The RECOMMENDER_TOP_N env variable sets the top-n option (default to 5).
  • The Reviewer Recommender server authenticates requests using the CI_JOB_TOKEN passed by the reviewers-recommender job automatically.
  • Our Applied ML group will automatically track the generated artifacts files and existing assignments to improve the quality of recommendations.
  • The reviewers-Recommender job is configured with the allow_failure: true parameter to prevent breaking the project pipeline.
  • Invite @GitLabReviewerRecommenderBot to the project with the Reporter permissions in order to get recommendations.

Known limitations:

  • The Reviewer Recommender may provide outdated recommendations. We are already working on this issue in gitlab#340459.
  • Our Applied ML group work to improve the way we present recommendations to MR authors.

You can find an example of how Reviewer Recommender works in this MR.

Related issue: https://gitlab.com/gitlab-org/gitlab/-/issues/342778, gitlab-org/modelops/applied-ml/review-recommender/recommender-bot-service#5

Edited by Alexander Chueshev

Merge request reports