Get familiar with UX of suggested reviewers
This issue is a follow-up from a meeting between the Code Review and Applied ML groups on 2022-06-08 (agenda, recording).
Purpose
Get familiar with:
- Current state of the experience of suggested reviewers, including:
- Internal and external (early beta customers) feedback.
- How suggestions are presented to users.
- How the feature is set up.
- Inner-workings of the model used for suggestions.
- Future plans for improvement.
- https://gitlab.com/gitlab-org/modelops/applied-ml/review-recommender/recommendation-engine/-/issues/4
- There may be other docs/issues with plans.
Compare with the competitor evaluation done in https://gitlab.com/gitlab-org/competitor-evaluations/-/issues/27+.
- Look at set up/configuration of other tools
Goals
- Understand the current/future state of the feature.
- Plan how UX can best support the feature's development.
- Highlight UX issues to resolve before general availability of the feature (must haves and nice to haves).
Summary
- Model v2 will be a vast improvement over the current iteration. It will take into account 11 additional inputs spread across two milestones; additionally, it should be quicker to update (faster pipelines)
- We 15 beta testers for the next iteration. They don't want to test the feature until it appears in the dropdown (they don't want the noise of the bot comments)
- Applied ML will be monitoring the model accuracy ongoing as well as seeking out and exploring any feedback from the feature. They will also likely do some sort of running report on some recurring timeframe as it will directly drive model improvements.
- We do not need a setting to turn on/off the feature yet since the beta users will have to configure/opt-in
Edited by Annabel Dunstone Gray