Integrate Reviewer Recommender via the CI template for gitlab-org
What does this MR do and why?
Reviewer Recommender (aka UnReview) is an ML-based code reviewer recommendation system acquired by GitLab. By integrating a beta version of Reviewer Recommender into the GitLab project, our Applied ML group would like to collect feedback to improve the quality of recommendations and help the team control the review workload.
This MR allows getting code reviewer recommendations when running the reviewers-recommender
job included via the CI template. The reviewers-recommender
job generates an artifact file with top-n
recommendations, which can be used as a suggestion for assigning code reviewers. Reviewer-Recommender now also puts recommendations into an MR note similar to Dangerbot. At each CI job run, Reviewer-Recommender searches for its previous comment within the first 10 notes. If the comment is found, then Reviewer-Recommender updates it. If not, then Reviewer-Recommender creates a new comment. Multiple Reviewer-Recommender comments have a very low probability of appearing in newly created MRs. In general, this logic is implemented to limit the number of heavy API GitLab calls and will be updated soon.
Known limitations:
- The Reviewer Recommender may provide outdated recommendations. We are already working on this issue in #340459.
- Our Applied ML group work to improve the way we present recommendations to MR authors - https://gitlab.com/gitlab-org/modelops/applied-ml/review-recommender/recommender-bot-service/-/issues/6
Screenshots or screen recordings
You can find an example of how Reviewer Recommender works in this MR.
Please, also find other projects connected to Reviewer-Recommender:
- https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/4096
- gitlab-org/security-products/analyzers/semgrep!104 (merged)
- https://gitlab.com/gitlab-services/version-gitlab-com/-/merge_requests/633
How to set up and validate locally
You can include the reviewers-recommender
job via the CI template - https://gitlab.com/gitlab-org/modelops/applied-ml/review-recommender/ci-templates/-/blob/v0.2.1/recommender/Reviewers.gitlab-ci.yml. For now, we are keeping the ci-template project private and available for internal use only.
Configuration:
- The
reviewers-recommender
job doesn't change the existing review workflow. - Reviewer Recommender periodically collects the merge request history and trains the model. Please, find the GitLab scheduled pipeline here (internal only): https://gitlab.com/gitlab-org/modelops/applied-ml/review-recommender/pipeline-scheduler/-/pipeline_schedules
- The
RECOMMENDER_TOP_N
env variable sets thetop-n
option (default to 5). - The Reviewer Recommender server authenticates requests using the
CI_JOB_TOKEN
passed by thereviewers-recommender
job automatically. - Our Applied ML group will automatically track recommendations and existing assignments to improve the quality of recommendations.
- The
reviewers-Recommender
job is configured with theallow_failure: true
parameter to prevent breaking the project pipeline. -
Invite @GitLabReviewerRecommenderBot to the project with the(not required for this project)Reporter
permissions in order to get recommendations.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.