Skip to content

Integrate Reviewer Recommender via the CI template for gitlab-org

Alexander Chueshev requested to merge ci-reviewer-recommender into master

What does this MR do and why?

Reviewer Recommender (aka UnReview) is an ML-based code reviewer recommendation system acquired by GitLab. By integrating a beta version of Reviewer Recommender into the GitLab project, our Applied ML group would like to collect feedback to improve the quality of recommendations and help the team control the review workload.

This MR allows getting code reviewer recommendations when running the reviewers-recommender job included via the CI template. The reviewers-recommender job generates an artifact file with top-n recommendations, which can be used as a suggestion for assigning code reviewers. Reviewer-Recommender now also puts recommendations into an MR note similar to Dangerbot. At each CI job run, Reviewer-Recommender searches for its previous comment within the first 10 notes. If the comment is found, then Reviewer-Recommender updates it. If not, then Reviewer-Recommender creates a new comment. Multiple Reviewer-Recommender comments have a very low probability of appearing in newly created MRs. In general, this logic is implemented to limit the number of heavy API GitLab calls and will be updated soon.

Known limitations:

Screenshots or screen recordings

You can find an example of how Reviewer Recommender works in this MR.

Please, also find other projects connected to Reviewer-Recommender:

How to set up and validate locally

You can include the reviewers-recommender job via the CI template - https://gitlab.com/gitlab-org/modelops/applied-ml/review-recommender/ci-templates/-/blob/v0.2.1/recommender/Reviewers.gitlab-ci.yml. For now, we are keeping the ci-template project private and available for internal use only.

Configuration:

  • The reviewers-recommender job doesn't change the existing review workflow.
  • Reviewer Recommender periodically collects the merge request history and trains the model. Please, find the GitLab scheduled pipeline here (internal only): https://gitlab.com/gitlab-org/modelops/applied-ml/review-recommender/pipeline-scheduler/-/pipeline_schedules
  • The RECOMMENDER_TOP_N env variable sets the top-n option (default to 5).
  • The Reviewer Recommender server authenticates requests using the CI_JOB_TOKEN passed by the reviewers-recommender job automatically.
  • Our Applied ML group will automatically track recommendations and existing assignments to improve the quality of recommendations.
  • The reviewers-Recommender job is configured with the allow_failure: true parameter to prevent breaking the project pipeline.
  • Invite @GitLabReviewerRecommenderBot to the project with the Reporter permissions in order to get recommendations. (not required for this project)

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Alexander Chueshev

Merge request reports