Update feedback mechanism in the chat

Context

We currently have the following feedback mechanism in the chat:

UX has an initiative to standardize the feedback mechanism for AI features. Three designs were tested with users to arrive at the current solution. You can read about the rationale for changing the current implementation & the user testing findings in the Pajamas issue

Problem to solve

These findings are informed by UX research. Read more here.

• Users are motivated to give feedback when they believe the feedback will improve the AI's output
• Users want to provide context with their feedback
• Users did not believe clicking helpful, unhelpful, wrong would lead to any follow-up questions
• Users found it difficult to categorize between helpful, unhelpful, wrong

Proposal

Requirements

• Each AI response has a call to action under the content
• The CTA links to a modal
• When the modal is submitted, the user sees a confirmation where the original CTA was
• When the modal is submitted, a toast is shown and dismisses after a set time

Nice to have

• The modal is going to be the same modal across different AI features. It would be nice if it could be built once and re-used by other teams.

Analytics questions

How common it is to give feedback?

Of the total answers given by GitLab Duo, how many of them receive feedback?

How common it it to abandon the feedback form?

Of the total times the feedback form is opened, how often is it submitted?

Which tool's answers receive the most feedback?

I'm interested to learn if certain tools receive the most feedback. Example output:

Feedback in the last 30 days

Tool	Feedback	Feedback type	Feedback verbatim
Tool 1	120	Unhelpful or irrelevant (40%), factually incorrect (20%)...	"the answer is wrong because...", "I'm giving feedback..."

Which users give the most feedback?

We could use their user ID to contact them to see if they want to speak with us

What we need to measure

• Total number of answers in GitLab Duo
  • Time answer was given
• Feedback form submitted
  • Time feedback was given
  • Tool used to generate answer
  • User ID of feedback giver
  • What was marked on the checkboxes
  • What was written in the free text
• Feedback form opened (total amount)