## Summary This issue tracks the rollout of the `slack_duo_agent` feature flag on production. The feature, introduced in [!225325](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/225325), enables handling of Slack `app_mention` events so that users can `@mention` the GitLab Duo bot in Slack. This is the first step toward a fully Duo Agent-powered Slack bot. The Duo Workflow integration will be wired in follow-up MRs to make this a proper MVC. Related feature issue: [#590434](https://gitlab.com/gitlab-org/gitlab/-/work_items/590434) ## Owners - Most appropriate Slack channel to reach out to: `#g_agent_foundations` - Best individual to reach out to: [@thomas-schmidt](https://gitlab.com/thomas-schmidt) ## Expectations **What are we expecting to happen?** Users who have the `slack_duo_agent` flag enabled can `@mention` the GitLab Duo bot in a Slack workspace that has the GitLab Slack integration installed. The bot handles the mention event end-to-end: - **Unauthenticated users** receive a :lock: reaction and an ephemeral prompt to connect their GitLab account. - **Authenticated users without the flag** receive a :lock: reaction and an ephemeral "you do not have access" message. - **Authenticated users with the flag** receive a :eyes: reaction, a reply in the thread echoing their message, and a :white_check_mark: reaction on completion. In follow-up MRs, the bot will be wired to Duo Agent Platform to allow users to search issues, MRs, create issues from Slack threads, and eventually trigger full agentic workflows — all without leaving Slack. **What can go wrong and how would we detect it?** - **Slack event processing failures**: The `SlackEventWorker` could fail to process `app_mention` events. Monitor Sidekiq error rates and the `integrations/slack_events/app_mentioned_service` error logs. - **Authentication flow breakage**: The `ChatName` lookup or OAuth link could fail, leaving users unable to connect their GitLab account. Watch for errors in `ChatNames::FindUserService`. - **Slack API call failures**: Calls to the Slack API (reactions, ephemeral messages, thread replies) could fail due to rate limiting or token issues. Monitor for `Slack::API` errors in logs. - **No data loss risk** for this MVC — the feature only reads Slack events and writes reactions/messages back to Slack. No GitLab data is mutated in this initial version. - **No impact on existing Slack integration users** — the feature is gated per-user and uses a separate Slack app manifest; existing slash command users are unaffected. **Which dashboards are most relevant?** - Sidekiq worker error rates: https://dashboards.gitlab.net (filter by `SlackEventWorker`) - Rails error rates and logs for `integrations/slack_events` ## Rollout Steps ### Rollout on non-production environments - [ ] Verify the MR with the feature flag is merged to `master` and has been deployed to non-production environments with `/chatops run auto_deploy status <merge-commit-of-your-feature>` - [ ] Enable the feature for specific internal test users on non-production environments: `/chatops run feature set --user=thomas-schmidt slack_duo_agent true --dev --pre --staging --staging-ref` - [ ] Verify the bot responds correctly to `@mentions` in a test Slack workspace connected to staging. - [ ] Enable the feature globally on non-production environments with: `/chatops run feature set slack_duo_agent true --dev --pre --staging --staging-ref` - [ ] Verify that the feature works as expected. The best environment to validate in is `staging-canary`. - [ ] If the feature flag causes end-to-end tests to fail, disable the feature flag on staging to avoid blocking deployments. See `#e2e-run-staging` Slack channel. ### Before production rollout - [ ] Announce in `#whats-happening-at-gitlab` if desired, before rollout to `gitlab-org`/`gitlab-com`. ### Specific rollout on production For visibility, all `/chatops` commands that target production must be executed in the `#production` Slack channel and cross-posted to `#g_agent_foundations`. - [ ] Ensure that the feature MRs ([!225325](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/225325) and follow-ups) have been deployed to both production and canary with `/chatops run auto_deploy status <merge-commit>` - [ ] Start with a small set of internal users (e.g. the Agent Foundations team): `/chatops run feature set --user=thomas-schmidt slack_duo_agent true` - [ ] Verify the bot works end-to-end in the internal GitLab Slack workspace. - [ ] Expand to all GitLab team members: `/chatops run feature set --feature-group=gitlab_team_members slack_duo_agent true` - [ ] Verify that the feature works for the expanded actor set. ### Preparation before global rollout - [ ] Set a milestone to this rollout issue to signal for enabling and removing the feature flag when it is stable. - [ ] Check if the feature flag change needs to be accompanied with a change management issue. - [ ] Ensure that [@thomas-schmidt](https://gitlab.com/thomas-schmidt) or a representative from `#g_agent_foundations` can be available for at least 2 hours after feature flag updates in production. - [ ] Ensure that documentation exists for the feature and the version history text has been updated. - [ ] Notify `#support_gitlab-com` and `#g_agent_foundations` before broader rollout. - [ ] Note: Since this is a `wip` type flag and currently only echoes back the user's message (no Duo Workflow wired yet), global rollout should wait until the Duo Agent integration follow-up MRs are merged and validated. ### Global rollout on production For visibility, all `/chatops` commands that target production must be executed in the `#production` Slack channel and cross-posted to `#g_agent_foundations`. - [ ] Incrementally roll out the feature on production: `/chatops run feature set slack_duo_agent <rollout-percentage> --actors` - [ ] Between every step wait for at least 15 minutes and monitor https://dashboards.gitlab.net. - [ ] Enable the feature globally on production: `/chatops run feature set slack_duo_agent true` - [ ] Wait for at least one day before releasing the feature. ### Release the feature - [ ] Create a merge request to remove the `slack_duo_agent` feature flag, including: - Remove all references to the feature flag from the codebase. - Remove `config/feature_flags/wip/slack_duo_agent.yml`. - Add a changelog entry. - [ ] Ensure the cleanup MR has been included in the release package. - [ ] Close [#590434](https://gitlab.com/gitlab-org/gitlab/-/work_items/590434) to indicate the feature will be released in the current milestone. - [ ] Clean up the feature flag from all environments: `/chatops run feature delete slack_duo_agent --dev --pre --staging --staging-ref --production` - [ ] Close this rollout issue. ## Rollback Steps - Disable the feature flag on production: `/chatops run feature set slack_duo_agent false` - Disable on non-production environments: `/chatops run feature set slack_duo_agent false --dev --pre --staging --staging-ref` - Delete feature flag from all environments: `/chatops run feature delete slack_duo_agent --dev --pre --staging --staging-ref --production`