Tracking user interaction with new homepage design

Following up from the direction proposed at #430735 (comment 2000213599)

Description

We are going to release an experiment for the MR homepage. To simplify the release, we are leaving out-of-scope filtering and search.

Button for switching the experiment: We will implement a button that allows users to switch the experiment on and off, enabling them to use the filtering and search features that are currently out of scope.

When seeing the old design When seeing the new design
Settings_dropdown-new.png Settings_dropdown-old.png

Implementation Plan

To ensure we gather useful data that will help us to improve the designs further, we will implement key actions:

  1. Track clicks: We will implement click tracking for users switching back to the old homepage. This will not provide detailed insights into the reasons behind the switch but will give us an indication that further investigation is needed.
  2. Feedback prompt: We will ask for feedback when users switch back to the old homepage, this might introduce friction, but we want feedback in the early days.
  3. Allow switching back to new design: Users will be able to switch back to the new design from the old design. This can be done through the new setting button, keeping the process simple and user-friendly.
  4. Directing users to the new experience: Users who turn off the experiment will be directed to the new experiment page when they click on 'My merge requests'. We do not want users to opt-out of the new experience entirely; instead, we aim to support a path for them to access the filtering and search.

Click tracking and metrics

To effectively track user interaction and gather meaningful insights, we recommend tracking the following metrics:

  1. Switch back to the old homepage:
    • Metric: Number of users clicking the switch back button.
    • Goal: Identify the volume of users opting out of the new homepage design.
  2. Switch back to new design:
    • Metric: Number of users switching back to the new design after opting out.
    • Goal: Measure the effectiveness and usability of the switching mechanism.
  3. Time spent on new homepage:
    • Metric: Average time users spend on the new homepage before switching back.
    • Goal: Understand user engagement and identify potential pain points.
Edited by Alex Fracazo