Deliver long Duo answers to Slack and report failed deliveries

What does this MR do and why?

GitLab Duo in Slack lost long answers. The session finished, but Slack rejected the final message and the user was left with a spinner that never stopped. Nothing in Slack said anything had gone wrong.

Cause: when Duo edits its progress message into the final answer, Slack limits the message's text field to 4,000 bytes. Slack counts that limit after it escapes &, < and >. Our code cut the text at 4,000 characters, so an answer with em dashes or code went over the limit. The first edit and the plain-text fallback sent the same oversized text, so both failed with msg_too_long, all retries failed the same way, and nobody was told.

This MR:

• Cuts the text field by escaped bytes, so an edited answer always fits. The answer itself sits in a markdown block, which holds up to 12,000 characters, so the full answer still shows.
• Shortens answers over 12,000 characters at a paragraph break and adds a note: "Answer shortened to fit in Slack. The full answer is in the GitLab session." The reply keeps its formatting and plan instead of falling back to cut, unformatted text.
• Adds a plain-text fallback to new replies. If Slack rejects the blocks for another reason, the answer goes out as plain text instead of failing.
• Tells the user when delivery fails for good. After the last retry, CallbackWorker adds ❌ to the user's message and replaces the spinner with "The task completed, but I couldn't post the answer here. Check the session in GitLab for the full response."

How we found it

• The production session finished normally. Duo Workflow Service logged workflow execution success, and the final answer was in the last checkpoint: 5,245 characters.

• Kibana showed Ai::Messaging::CallbackWorker failing 4 times, about 40 seconds apart. Each time, the block edit and the plain-text fallback both got msg_too_long, and then the job logged Duo Messaging: retries exhausted; giving up.

• We reproduced it against a test workspace. The production payload fails. The same blocks with a short text succeed. Slack's limits, measured:

  chat.update with	Largest accepted
  text, ASCII	4,000 characters
  text with em dashes	3,500 characters (4,000 bytes)
  text with & < >	2,333 characters (4,000 bytes after escaping)
  one markdown block	12,000 characters

• New replies (chat.postMessage) accepted the same 5.6k answer, so only the edit path broke.

References

• Slack thread where the answer got lost (internal): https://gitlab.slack.com/archives/C0BP2BL7TSB/p1790782312200109?thread_ts=1790703760.919779&cid=C0BP2BL7TSB
• Prompt change that asks for short Slack answers: !258899 (merged)
• Follow-up for session links on group-level sessions, which the new error message points to: #631621

Screenshots or screen recordings

Before	After
Production: the answer never arrives	The same answer arrives, with the plan marked complete
Answers over 12,000 characters were lost too	Shortened at a paragraph break, with a note
A failed delivery left the spinner running forever	The user is told, and the message gets ❌

How to set up and validate locally

You need a local Slack app connected to GDK, with Duo enabled for your user and the slack_duo_api_flow feature flag turned on. See Duo in Slack.

1. Check out this branch and restart Sidekiq, which runs the delivery: gdk restart rails-background-jobs.

2. In a channel with the app, ask for a long answer with a todo list. The todo list makes Duo post a progress message, and editing that message is the path that broke:

   @gitlab make a todo list with three steps, then write a detailed comparison of about 5,000 characters, at most 7,000, of three caching strategies for a Rails app. Use em dashes (—) a lot, bold labels and bullet lists. No headings.

3. Check that the progress message turns into Plan complete followed by the full answer, and that 👀 becomes ✅ on your message.

4. Run grep msg_too_long log/integrations_json.log and confirm no new entries appear.

5. Optional: to see the bug, switch to master, restart Sidekiq, and send the same message. The progress message stays stuck at "Working on your request… (3/3 done)".

Without an LLM: replay script that covers every path, including the failure message

Save this as slack_delivery_review.rb in your GitLab checkout and run it with your Slack workspace and channel IDs:

SLACK_TEAM_ID=T… SLACK_CHANNEL_ID=C… RUN_LABEL=branch bin/rails runner slack_delivery_review.rb

Set RUN_LABEL to master or branch so the Slack threads show which code they came from.

It creates a throwaway session, posts four threads, then deletes the session. Expected output:

Case	master	This branch
1. Long answer replaces the progress message	delivered=false, spinner stuck	delivered=true
2. Long answer posted as a new reply	delivered=true	delivered=true
3. Answer over 12,000 characters	delivered=false	delivered=true, shortened with a note
4. Delivery gives up after retries	user told=false	user told=true, ❌ and an error message

# Replays long Duo answers through the Slack adapter's delivery paths.
# Usage: SLACK_TEAM_ID=T… SLACK_CHANNEL_ID=C… [RUN_LABEL=master] bin/rails runner slack_delivery_review.rb
# The app's bot must be a member of the channel.
team_id = ENV.fetch('SLACK_TEAM_ID')
channel_id = ENV.fetch('SLACK_CHANNEL_ID')
api = Slack::API.new(SlackIntegration.with_bot.find_by!(team_id: team_id))
user_id = ChatName.where(team_id: team_id).pick(:chat_id)

section = "**Option — %<n>d** keeps the plan & stops the spinner — the title reads `Plan complete` " \
  "once every task is done.\n\n- Pro: no false checkmarks <-- important\n- Con: the last task disappears\n\n"
answer = "Here's my proposal — short version first.\n\n" + (1..30).map { |n| format(section, n: n) }.join +
  "```ruby\nsettled = todos.map { |t| t } # <-- uses settled\n```"
huge_answer = ([answer] * 3).join("\n\n")
puts "answers: #{answer.size} and #{huge_answer.size} chars"

workflow = Ai::DuoWorkflows::Workflow.create!(
  user: User.admins.first, project: Project.first, workflow_definition: 'slack_assistant/v1',
  goal: 'Slack delivery review', environment: :ambient
)

begin
  label = ENV['RUN_LABEL'].present? ? "[#{ENV['RUN_LABEL']}] " : ''
  new_thread = ->(title) { api.post_message(channel: channel_id, text: "#{label}#{title}")['ts'] }
  progress_message = ->(thread_ts) do
    api.post_message(channel: channel_id, thread_ts: thread_ts, text: 'Working on your request…')['ts']
  end
  ctx_for = ->(thread_ts, status_ts = nil) do
    { 'adapter' => 'slack', 'team_id' => team_id, 'channel_id' => channel_id, 'thread_ts' => thread_ts,
      'message_ts' => thread_ts, 'user_id' => user_id, 'workflow_id' => workflow.id,
      'status_ts' => status_ts }.compact
  end
  deliver = ->(ctx, message) do
    Ai::Messaging::Adapters::Slack.from_callback_context(ctx)
      .deliver_result(callback_context: ctx, message: message, workflow: workflow)
  end

  t = new_thread.call('Review 1/4: long answer replaces the progress message')
  puts "1 edit path: delivered=#{deliver.call(ctx_for.call(t, progress_message.call(t)), answer)}"

  t = new_thread.call('Review 2/4: long answer posted as a new reply')
  puts "2 post path: delivered=#{deliver.call(ctx_for.call(t), answer)}"

  t = new_thread.call('Review 3/4: answer over 12,000 characters is shortened, keeping its formatting')
  puts "3 over 12k: delivered=#{deliver.call(ctx_for.call(t, progress_message.call(t)), huge_answer)}"

  t = new_thread.call('Review 4/4: delivery gives up after retries, so the user is told')
  workflow.update!(messaging_callback_context: ctx_for.call(t, progress_message.call(t)))
  Ai::Messaging::CallbackWorker.sidekiq_retries_exhausted_block.call(
    { 'args' => ['Ai::DuoWorkflows::WorkflowFinishedEvent', { 'workflow_id' => workflow.id }] },
    Ai::Messaging::ResultDelivery::DeliveryRetryError.new
  )
  puts "4 give-up: user told=#{workflow.reload.messaging_callback_context['delivered_at'].present?}"
ensure
  workflow.destroy!
end

Known limitations

• Answers over 12,000 characters lose their ending. The cut happens at a paragraph break and the note tells the user, but the rest is only in the session. In local testing, a request for 5,000 characters produced 11,085 and 12,022 characters, so this happens in practice. We plan to propose posting the rest as follow-up messages instead.
• The error message mentions the session but has no link for group-level sessions, which includes Slack sessions. #631621 tracks that.
• The ❌ appears after about 2 minutes, because errors that won't change on retry are still retried 3 times. Planned as a follow-up.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.