<details> <summary> Everyone can contribute. [Help move this issue forward](https://handbook.gitlab.com/handbook/marketing/developer-relations/contributor-success/community-contributors-workflows/#contributor-links) while earning points, leveling up and collecting rewards. </summary> - [Label this issue](https://contributors.gitlab.com/manage-issue?action=label&projectId=278964&issueIid=603660) </details> ## Summary Provide a low-friction way for users to give feedback in Slack on whether a GitLab Duo agent interaction was helpful or not, as an additional quality signal. ## Problem Today, feedback on Duo agent interactions through Slack is collected informally via a [feedback issue](https://gitlab.com/gitlab-org/gitlab/-/work_items/602775) used for internal testing. There is no low-friction, in-context way for users to signal whether an agent's response was helpful. We already have thumbs up/down feedback on other surfaces (e.g. code review), but this is not carried through to Slack interactions. ## Proposal > _The exploration of options and the pseudo-code below were AI-assisted (researched with GitLab Duo Web Chat)._ **Approach:** Block Kit `feedback_buttons` for the UX + the already-wired Slack interactions endpoint for handling + a modal on 👎 to capture a reason + the shared `ai_usage_events` schema for storage. ### Why this fits cleanly - The Slack interactions path is **already wired**: `API::Integrations::Slack::Interactions` → `Integrations::SlackInteractionService` receives `block_actions` payloads. No new event subscription or OAuth scope (unlike `reaction_added`, which would also need `ts`-to-session correlation). - We reuse the **existing AI usage tracking schema** (the `ai_usage_events` record): `event`, `label`, `property`, `value`, `extras` (jsonb), `user_id`, `namespace_id`, `timestamp`. Only the `event` name is feature-specific so metrics can slice on it; surface lives in `property` so the event generalizes beyond Slack (e.g. Teams later). This mirrors the agentic-chat `ai_duo_agentic_chat_feedback_submitted` pattern. ### 1. Post feedback buttons with the Duo response In `ee/app/services/ai/messaging/adapters/slack.rb`, append a `context_actions` block when delivering the result. Pack the workflow/session ID into the button `value` so the click is self-contained (no `ts`-to-session lookup needed): ```ruby def feedback_block(workflow) { type: 'context_actions', elements: [{ type: 'feedback_buttons', action_id: 'duo_feedback', positive_button: { text: { type: 'plain_text', text: '👍' }, value: "up:#{workflow.id}" }, negative_button: { text: { type: 'plain_text', text: '👎' }, value: "down:#{workflow.id}" } }] } end ``` ### 2. Handle the click via the existing interactions path Add a branch for our `action_id` in `SlackInteractionService`: ```ruby def route_interaction action = params.dig(:actions, 0) return handle_duo_feedback(action) if action[:action_id] == 'duo_feedback' # ... existing routing end def handle_duo_feedback(action) direction, workflow_id = action[:value].split(':') # "up" / "down" if direction == 'down' open_feedback_modal(trigger_id: params[:trigger_id], workflow_id: workflow_id) else track_feedback(direction: direction, workflow_id: workflow_id) end end ``` On 👎, open a reason modal with `views.open` using `trigger_id`; on 👍, record and ack immediately. ### 3. The 👎 reason modal We collect more detail only when the response was graded poorly (low friction on positive feedback). Lean set of reasons tailored to our Slack Duo agent surface (which performs actions, not just Q&A): - Incorrect or misleading - Didn't do what I asked - Incomplete - Other Plus an optional free-text "Tell us more" field and a short data-sharing disclaimer (e.g. "This feedback helps us improve Duo. We'll share your message and recent thread context with GitLab."). The selected reason + free text feed into the event's `property` / `extras`. ### 4. Track via the shared schema Register a new event in `Gitlab::Tracking::AiTracking` so it dual-writes to the `ai_usage_events` Postgres + ClickHouse tables and flows into the AI Impact dashboards. Reuse the standard fields, no custom schema: ```ruby def track_feedback(direction:, workflow_id:, reason: nil) track_internal_event( 'ai_duo_messaging_feedback_submitted', user: current_user, additional_properties: { label: direction == 'up' ? 'thumbs_up' : 'thumbs_down', # label value: workflow_id.to_i, # value property: 'slack', # surface (generalizes to Teams later) reason: reason # → extras jsonb } ) end ``` Event definition (`ee/config/events/ai_duo_messaging_feedback_submitted.yml`) mirrors `ai_duo_agentic_chat_feedback_submitted.yml`: `internal_events: true`, `product_group: duo_chat`, tiers premium/ultimate, with `label`/`value`/`property` documented under `additional_properties`. ### Decisions for this first step - **Re-clicks:** we just **append** for now (simpler). `ai_usage_events` is `ReplacingMergeTree` keyed on `(namespace_path, event, timestamp, user_id)`, so vote changes land as separate rows; we can revisit dedupe later if needed. - **Block availability:** `feedback_buttons` / `context_actions` are newer AI-app block types. We'll verify they render across channel / thread / DM during testing. ### Out of scope (for now) - Plain `actions` + `button` fallback (we'll figure out the need during testing). - Delete `icon_button` on messages (orthogonal to feedback). - Sentiment parsing of thread follow-ups (the stretch goal below). ### Sequencing 1. Post buttons. 2. Handle 👍 + tracking. 3. Add the 👎 reason modal. ### Future / stretch - (Stretch) Explore sentiment parsing of follow-up replies in the thread as an additional signal. - (Future) Once source tracing for AI-generated content lands, consider metrics on follow-up actions (e.g. do issues created via Slack get closed again, do MRs get merged, etc.) as a stronger downstream signal of helpfulness. ## Related - Parent epic: &22305 (GitLab Duo Slack Integration: Customer Rollout) - Internal MVC: #590434 - Current feedback issue: #602775