[FF] `ai_use_messaging_adapter_for_mentions` -- Use messaging adapter for mention

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

Summary

This issue is to roll out the feature on production, that is currently behind the ai_use_messaging_adapter_for_mentions feature flag.

When enabled, foundational-flow @mention triggers are routed through the GitlabDuoNote messaging adapter instead of the legacy FlowTriggers::CreateNoteService path. The adapter delivers the same UX as @GitLabDuo:

  • an inline "started session" system note posted in the mention thread,
  • a framework-delivered reply when the workflow finishes (posted by Ai::Messaging::CallbackWorker),
  • correctly-branded activity-log session notes (authored by the flow-trigger service account).

This replaces today's self-posted 🔄 Processing… → ✅ has started comment where the agent posts its own reply.

Scope: routing applies to any foundational flow wired as a :mention trigger (Duo Developer / developer/v1 is the primary current consumer; others such as fix_pipeline/v1 qualify if mention-triggered). Non-foundational mention triggers and non-mention triggers (:assign, :pipeline) stay on the legacy RunService path, unchanged. @GitLabDuo MR mentions use a separate handler and are not affected by this flag.

  • Feature flag: ai_use_messaging_adapter_for_mentions
  • Actor: project (Feature.enabled?(:ai_use_messaging_adapter_for_mentions, note.project))

Owners

  • Most appropriate Slack channel to reach out to: #g_agent_foundations
  • Best individual to reach out to: @thomas-schmidt

Expectations

What are we expecting to happen?

Users who @mention a foundational-flow service account on an issue, merge request, or work item see the new adapter UX: an inline started-session note immediately, then a framework-delivered reply in the same thread when the workflow completes, plus the standard activity-log session notes. Behavior for non-foundational and non-mention triggers is unchanged.

What can go wrong and how would we detect it?

  • No reply after the started note is destroyed (top risk). On completion, CallbackWorker#extract_final_message posts the last agent message from the workflow checkpoint. If a foundational flow produces no such message, the started note is destroyed and the user sees only a generic "no response" reply (or nothing). Validate end-to-end on a live flow early in rollout. Detect via: user reports, Ai::Messaging::CallbackWorker exceptions in Sentry, and Sidekiq metrics for feature_category=duo_agent_platform.
  • Service-account authoring/permissions. Started/reply notes are authored as the flow-trigger service account, pre-provisioned as a project member at consumer creation. If membership is missing, create_note_on degrades silently (no note). Detect: missing replies + Sentry.
  • Shared adapter refactor shipped un-flagged. The same MR refactored the shared, ungated adapter path (Base#trigger → with_lifecycle_hooks, the GitlabDuoNote constructor). Those changes affect @GitLabDuo MR mentions and Slack regardless of this flag — a regression there would not be controlled by toggling it. Watch @GitLabDuo reply behavior during the same window.

Relevant dashboards on https://dashboards.gitlab.net: the sidekiq service filtered to Ai::Messaging::CallbackWorker / feature_category=duo_agent_platform, and the rails web error-rate / latency panels.

Rollout Steps

Note: Please make sure to run the chatops commands in the Slack channel that gets impacted by the command.

Rollout on non-production environments

  • Verify the MR with the feature flag is merged to master and has been deployed to non-production environments with /chatops gitlab run auto_deploy status <merge-commit-of-your-feature>

  • Deploy the feature flag at a percentage (recommended percentage: 50%) with /chatops gitlab run feature set ai_use_messaging_adapter_for_mentions 50 --actors --dev --pre --staging --staging-ref

  • Monitor that the error rates did not increase (repeat with a different percentage as necessary).

  • Enable the feature globally on non-production environments with /chatops gitlab run feature set ai_use_messaging_adapter_for_mentions true --dev --pre --staging --staging-ref

  • Verify that the feature works as expected. The best environment to validate the feature in is staging-canary as this is the first environment deployed to. Make sure you are configured to use canary.

  • If the feature flag causes end-to-end tests to fail, disable the feature flag on staging to avoid blocking deployments.

    • See #e2e-run-staging Slack channel and look for the following messages:
      • test kicked off: Feature flag ai_use_messaging_adapter_for_mentions has been set to true on **gstg**
      • test result: This pipeline was triggered due to toggling of ai_use_messaging_adapter_for_mentions feature flag

If you encounter end-to-end test failures and are unable to diagnose them, you may reach out to the #s_developer_experience Slack channel for assistance. Note that end-to-end test failures on staging-ref don't block deployments.

Before production rollout

  • If the change is significant and you wanted to announce in #whats-happening-at-gitlab, it best to do it before rollout to gitlab-org/gitlab-com.

Specific rollout on production

For visibility, all /chatops commands that target production must be executed in the #production Slack channel and cross-posted (with the command results) to the responsible team's Slack channel.

  • Ensure that the feature MRs have been deployed to both production and canary with /chatops gitlab run auto_deploy status <merge-commit-of-your-feature>
  • This flag uses a project actor — enable for specific projects first:
    • /chatops gitlab run feature set --project=gitlab-org/gitlab ai_use_messaging_adapter_for_mentions true
  • Verify that the feature works for the specific actors.

Preparation before global rollout

  • Set a milestone to this rollout issue to signal for enabling and removing the feature flag when it is stable.
  • Check if the feature flag change needs to be accompanied with a change management issue. Cross link the issue here if it does.
  • Ensure that you or a representative in development can be available for at least 2 hours after feature flag updates in production. If a different developer will be covering, or an exception is needed, please inform the oncall SRE by using the @sre-oncall Slack alias.
  • Ensure that documentation exists for the feature, and the version history text has been updated.
  • Ensure that any breaking changes have been announced following the release post process to ensure GitLab customers are aware.
  • Notify the #support_gitlab-com Slack channel and your team channel (more guidance when this is necessary in the dev docs).

Global rollout on production

For visibility, all /chatops commands that target production must be executed in the #production Slack channel and cross-posted (with the command results) to the responsible team's Slack channel.

Release the feature

After the feature has been deemed stable, the clean up should be done as soon as possible to permanently enable the feature and reduce complexity in the codebase.

  • Create a merge request to remove the ai_use_messaging_adapter_for_mentions feature flag. Ask for review/approval/merge as usual. The MR should include the following changes:
    • Remove all references to the feature flag from the codebase.
    • Remove the YAML definitions for the feature from the repository.
  • Ensure that the cleanup MR has been included in the release package. If the merge request was deployed before the monthly release was tagged, the feature can be officially announced in a release blog post: /chatops gitlab run release check <merge-request-url> <milestone>
  • Close the feature issue to indicate the feature will be released in the current milestone.
  • Once the cleanup MR has been deployed to production, clean up the feature flag from all environments by running these chatops command in #production channel: /chatops gitlab run feature delete ai_use_messaging_adapter_for_mentions --dev --pre --staging --staging-ref --production
  • Close this rollout issue.

Rollback Steps

  • This feature can be disabled on production by running the following Chatops command:
/chatops gitlab run feature set ai_use_messaging_adapter_for_mentions false
  • Disable the feature flag on non-production environments:
/chatops gitlab run feature set ai_use_messaging_adapter_for_mentions false --dev --pre --staging --staging-ref
  • Delete feature flag from all environments:
/chatops gitlab run feature delete ai_use_messaging_adapter_for_mentions --dev --pre --staging --staging-ref --production
Edited by 🤖 GitLab Bot 🤖