Wire callback_hook_id into the create-flow API (!249147) · Merge requests · GitLab.org / GitLab

What does this MR do and why?

Adds callback_hook_id to POST /ai/duo_workflows/workflows, so a caller can attach a webhook to a flow when the flow starts. GitLab then POSTs lifecycle events to that webhook and the client does not have to poll.

This is the wiring half of Duo flow webhook callbacks. The delivery machinery lives in the MR this one targets and ships inert; nothing builds a callback context until this lands.

Ai::DuoWorkflows::CallbackHooksFinder resolves the hook. A hook qualifies when duo_flow_callback_enabled is set and it belongs to the flow's project or namespace, or to one of its ancestor groups. Group webhooks already fan out to descendant projects for every other event type, so this is the existing group-webhook semantic rather than a new permission surface. Authorization is containment: the caller can already run a flow here, and only a hook owner can set the flag. Group hooks stay behind the group_webhooks licence.

An unusable callback_hook_id returns 400 instead of a 201 with the callback quietly dropped, which would leave the client waiting for events that never arrive.

The feature flag is defined here

duo_flow_callback_hooks (wip, default off) is introduced by this MR, because subscribing a flow is its first real use. It is evaluated against the container's root ancestor, so there is one switch per hierarchy.

The flag decides which flows may subscribe, not which deliveries go out. The delivery path in !249145 (merged) does not read it: once a flow has subscribed, only the per-hook duo_flow_callback_enabled switch is re-read, so unticking a hook stops a running flow but flipping the flag does not. That trade-off is described in the delivery MR.

The hooks API and settings checkbox (!249149 (merged)) reads the flag too, and is stacked on this MR for the definition.

References

Issue: #606804 (closed)
Epic: gitlab-org#22921
Feature flag rollout: #608649
Split from !248060 (closed)

Screenshots or screen recordings

Not applicable, backend only.

How to set up and validate locally

Enable the flag for a top-level group:

Feature.enable(:duo_flow_callback_hooks, Group.find_by_full_path('my-group'))

Add a webhook on that group, or on a project inside it, with duo_flow_callback_enabled set:

GroupHook.create!(group: Group.find_by_full_path('my-group'),
  url: 'https://example.com/callback', duo_flow_callback_enabled: true)

Start a flow that references the hook:

curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \
  --url "https://gitlab.example.com/api/v4/ai/duo_workflows/workflows" \
  --data "project_id=<project_id>&callback_hook_id=<hook_id>&client_reference=my-run-1"

Confirm the hook was attached:

Ai::DuoWorkflows::Workflow.last.messaging_callback_context
# => { "adapter" => "webhook", "hook_id" => 1, "client_reference" => "my-run-1" }

Repeat step 3 with a hook id from an unrelated group and confirm the response is 400.
Disable the flag and repeat step 3 with the valid hook id: also 400, since the flag gates subscribing.

Database review

⚠️ Note: In order to test the below queries I needed to manually add the column first via exec since the migration was not reflected in the production db snapshot yet

To get the query I ran the following in the rails console:

project = Project.find(1000000)
ActiveRecord::Base.connection.unprepared_statement do
  puts Ai::DuoWorkflows::CallbackHooksFinder.new(container: project).execute.to_sql
end

And then replaced the project_id in the query with 278964 and the namespace with 9970 (as suggested by Mohamed here).

Query

SELECT "web_hooks"."id", "web_hooks"."project_id", "web_hooks"."created_at", "web_hooks"."updated_at", "web_hooks"."type", "web_hooks"."push_events", "web_hooks"."issues_events", "web_hooks"."merge_requests_events", "web_hooks"."tag_push_events", "web_hooks"."group_id", "web_hooks"."note_events", "web_hooks"."enable_ssl_verification", "web_hooks"."wiki_page_events", "web_hooks"."pipeline_events", "web_hooks"."confidential_issues_events", "web_hooks"."repository_update_events", "web_hooks"."job_events", "web_hooks"."confidential_note_events", "web_hooks"."push_events_branch_filter", "web_hooks"."encrypted_token", "web_hooks"."encrypted_token_iv", "web_hooks"."encrypted_url", "web_hooks"."encrypted_url_iv", "web_hooks"."deployment_events", "web_hooks"."releases_events", "web_hooks"."feature_flag_events", "web_hooks"."member_events", "web_hooks"."subgroup_events", "web_hooks"."recent_failures", "web_hooks"."disabled_until", "web_hooks"."encrypted_url_variables", "web_hooks"."encrypted_url_variables_iv", "web_hooks"."integration_id", "web_hooks"."branch_filter_strategy", "web_hooks"."emoji_events", "web_hooks"."name", "web_hooks"."description", "web_hooks"."custom_webhook_template", "web_hooks"."resource_access_token_events", "web_hooks"."encrypted_custom_headers", "web_hooks"."encrypted_custom_headers_iv", "web_hooks"."project_events", "web_hooks"."vulnerability_events", "web_hooks"."member_approval_events", "web_hooks"."milestone_events", "web_hooks"."organization_id", "web_hooks"."resource_deploy_token_events", "web_hooks"."filter", "web_hooks"."signing_token", "web_hooks"."duo_flow_callback_enabled" FROM ((SELECT "web_hooks"."id", "web_hooks"."project_id", "web_hooks"."created_at", "web_hooks"."updated_at", "web_hooks"."type", "web_hooks"."push_events", "web_hooks"."issues_events", "web_hooks"."merge_requests_events", "web_hooks"."tag_push_events", "web_hooks"."group_id", "web_hooks"."note_events", "web_hooks"."enable_ssl_verification", "web_hooks"."wiki_page_events", "web_hooks"."pipeline_events", "web_hooks"."confidential_issues_events", "web_hooks"."repository_update_events", "web_hooks"."job_events", "web_hooks"."confidential_note_events", "web_hooks"."push_events_branch_filter", "web_hooks"."encrypted_token", "web_hooks"."encrypted_token_iv", "web_hooks"."encrypted_url", "web_hooks"."encrypted_url_iv", "web_hooks"."deployment_events", "web_hooks"."releases_events", "web_hooks"."feature_flag_events", "web_hooks"."member_events", "web_hooks"."subgroup_events", "web_hooks"."recent_failures", "web_hooks"."disabled_until", "web_hooks"."encrypted_url_variables", "web_hooks"."encrypted_url_variables_iv", "web_hooks"."integration_id", "web_hooks"."branch_filter_strategy", "web_hooks"."emoji_events", "web_hooks"."name", "web_hooks"."description", "web_hooks"."custom_webhook_template", "web_hooks"."resource_access_token_events", "web_hooks"."encrypted_custom_headers", "web_hooks"."encrypted_custom_headers_iv", "web_hooks"."project_events", "web_hooks"."vulnerability_events", "web_hooks"."member_approval_events", "web_hooks"."milestone_events", "web_hooks"."organization_id", "web_hooks"."resource_deploy_token_events", "web_hooks"."filter", "web_hooks"."signing_token", "web_hooks"."duo_flow_callback_enabled" FROM "web_hooks" WHERE "web_hooks"."type" = 'ProjectHook' AND "web_hooks"."project_id" = 278964)
UNION ALL
(SELECT "web_hooks"."id", "web_hooks"."project_id", "web_hooks"."created_at", "web_hooks"."updated_at", "web_hooks"."type", "web_hooks"."push_events", "web_hooks"."issues_events", "web_hooks"."merge_requests_events", "web_hooks"."tag_push_events", "web_hooks"."group_id", "web_hooks"."note_events", "web_hooks"."enable_ssl_verification", "web_hooks"."wiki_page_events", "web_hooks"."pipeline_events", "web_hooks"."confidential_issues_events", "web_hooks"."repository_update_events", "web_hooks"."job_events", "web_hooks"."confidential_note_events", "web_hooks"."push_events_branch_filter", "web_hooks"."encrypted_token", "web_hooks"."encrypted_token_iv", "web_hooks"."encrypted_url", "web_hooks"."encrypted_url_iv", "web_hooks"."deployment_events", "web_hooks"."releases_events", "web_hooks"."feature_flag_events", "web_hooks"."member_events", "web_hooks"."subgroup_events", "web_hooks"."recent_failures", "web_hooks"."disabled_until", "web_hooks"."encrypted_url_variables", "web_hooks"."encrypted_url_variables_iv", "web_hooks"."integration_id", "web_hooks"."branch_filter_strategy", "web_hooks"."emoji_events", "web_hooks"."name", "web_hooks"."description", "web_hooks"."custom_webhook_template", "web_hooks"."resource_access_token_events", "web_hooks"."encrypted_custom_headers", "web_hooks"."encrypted_custom_headers_iv", "web_hooks"."project_events", "web_hooks"."vulnerability_events", "web_hooks"."member_approval_events", "web_hooks"."milestone_events", "web_hooks"."organization_id", "web_hooks"."resource_deploy_token_events", "web_hooks"."filter", "web_hooks"."signing_token", "web_hooks"."duo_flow_callback_enabled" FROM "web_hooks" WHERE "web_hooks"."type" = 'GroupHook' AND "web_hooks"."group_id" IN (SELECT "namespaces"."id" FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = 9970))) web_hooks WHERE "web_hooks"."duo_flow_callback_enabled" = TRUE

Here is the postgresAI session link where I ran the query.

Query 2 - nested project

In order to simulate a query with a more nested project, I used the following snippet

def sql_for(project_id:, group_id:, parent_id:, traversal_ids:)
  group   = Group.new(id: group_id, parent_id: parent_id, traversal_ids: traversal_ids)
  project = Project.new(id: project_id, group: group)

  # Without an EE licence the group leg silently disappears and the query is only half of the real one.
  unless group.licensed_feature_available?(:group_webhooks)
    abort 'ERROR: group_webhooks is not licensed locally; the GroupHook leg would be missing.'
  end

  ActiveRecord::Base.connection.unprepared_statement do
    Ai::DuoWorkflows::CallbackHooksFinder.new(container: project).execute.to_sql
  end
end

puts '=== 1. Root-level project: gitlab-org/gitlab (278964) under gitlab-org (9970) ==='
puts 'self_and_ancestors short-circuits to `id = 9970` because parent_id is nil.'
puts sql_for(project_id: 278964, group_id: 9970, parent_id: nil, traversal_ids: [9970])

puts
puts '=== 2. Nested project: gitlab-org/security-products/analyzers/semgrep (24075172) ==='
puts 'analyzers 2564205 -> security-products 2452873 -> gitlab-org 9970, so the'
puts 'lineage branch runs and renders `id IN (9970, 2452873, 2564205)`.'
puts sql_for(project_id: 24075172, group_id: 2564205, parent_id: 2452873,
  traversal_ids: [9970, 2452873, 2564205])

Which resulted in this query:

SELECT "web_hooks"."id", "web_hooks"."project_id", "web_hooks"."created_at", "web_hooks"."updated_at", "web_hooks"."type", "web_hooks"."push_events", "web_hooks"."issues_events", "web_hooks"."merge_requests_events", "web_hooks"."tag_push_events", "web_hooks"."group_id", "web_hooks"."note_events", "web_hooks"."enable_ssl_verification", "web_hooks"."wiki_page_events", "web_hooks"."pipeline_events", "web_hooks"."confidential_issues_events", "web_hooks"."repository_update_events", "web_hooks"."job_events", "web_hooks"."confidential_note_events", "web_hooks"."push_events_branch_filter", "web_hooks"."encrypted_token", "web_hooks"."encrypted_token_iv", "web_hooks"."encrypted_url", "web_hooks"."encrypted_url_iv", "web_hooks"."deployment_events", "web_hooks"."releases_events", "web_hooks"."feature_flag_events", "web_hooks"."member_events", "web_hooks"."subgroup_events", "web_hooks"."recent_failures", "web_hooks"."disabled_until", "web_hooks"."encrypted_url_variables", "web_hooks"."encrypted_url_variables_iv", "web_hooks"."integration_id", "web_hooks"."branch_filter_strategy", "web_hooks"."emoji_events", "web_hooks"."name", "web_hooks"."description", "web_hooks"."custom_webhook_template", "web_hooks"."resource_access_token_events", "web_hooks"."encrypted_custom_headers", "web_hooks"."encrypted_custom_headers_iv", "web_hooks"."project_events", "web_hooks"."vulnerability_events", "web_hooks"."member_approval_events", "web_hooks"."milestone_events", "web_hooks"."organization_id", "web_hooks"."resource_deploy_token_events", "web_hooks"."filter", "web_hooks"."signing_token", "web_hooks"."duo_flow_callback_enabled" FROM ((SELECT "web_hooks"."id", "web_hooks"."project_id", "web_hooks"."created_at", "web_hooks"."updated_at", "web_hooks"."type", "web_hooks"."push_events", "web_hooks"."issues_events", "web_hooks"."merge_requests_events", "web_hooks"."tag_push_events", "web_hooks"."group_id", "web_hooks"."note_events", "web_hooks"."enable_ssl_verification", "web_hooks"."wiki_page_events", "web_hooks"."pipeline_events", "web_hooks"."confidential_issues_events", "web_hooks"."repository_update_events", "web_hooks"."job_events", "web_hooks"."confidential_note_events", "web_hooks"."push_events_branch_filter", "web_hooks"."encrypted_token", "web_hooks"."encrypted_token_iv", "web_hooks"."encrypted_url", "web_hooks"."encrypted_url_iv", "web_hooks"."deployment_events", "web_hooks"."releases_events", "web_hooks"."feature_flag_events", "web_hooks"."member_events", "web_hooks"."subgroup_events", "web_hooks"."recent_failures", "web_hooks"."disabled_until", "web_hooks"."encrypted_url_variables", "web_hooks"."encrypted_url_variables_iv", "web_hooks"."integration_id", "web_hooks"."branch_filter_strategy", "web_hooks"."emoji_events", "web_hooks"."name", "web_hooks"."description", "web_hooks"."custom_webhook_template", "web_hooks"."resource_access_token_events", "web_hooks"."encrypted_custom_headers", "web_hooks"."encrypted_custom_headers_iv", "web_hooks"."project_events", "web_hooks"."vulnerability_events", "web_hooks"."member_approval_events", "web_hooks"."milestone_events", "web_hooks"."organization_id", "web_hooks"."resource_deploy_token_events", "web_hooks"."filter", "web_hooks"."signing_token", "web_hooks"."duo_flow_callback_enabled" FROM "web_hooks" WHERE "web_hooks"."type" = 'ProjectHook' AND "web_hooks"."project_id" = 24075172)
UNION ALL
(SELECT "web_hooks"."id", "web_hooks"."project_id", "web_hooks"."created_at", "web_hooks"."updated_at", "web_hooks"."type", "web_hooks"."push_events", "web_hooks"."issues_events", "web_hooks"."merge_requests_events", "web_hooks"."tag_push_events", "web_hooks"."group_id", "web_hooks"."note_events", "web_hooks"."enable_ssl_verification", "web_hooks"."wiki_page_events", "web_hooks"."pipeline_events", "web_hooks"."confidential_issues_events", "web_hooks"."repository_update_events", "web_hooks"."job_events", "web_hooks"."confidential_note_events", "web_hooks"."push_events_branch_filter", "web_hooks"."encrypted_token", "web_hooks"."encrypted_token_iv", "web_hooks"."encrypted_url", "web_hooks"."encrypted_url_iv", "web_hooks"."deployment_events", "web_hooks"."releases_events", "web_hooks"."feature_flag_events", "web_hooks"."member_events", "web_hooks"."subgroup_events", "web_hooks"."recent_failures", "web_hooks"."disabled_until", "web_hooks"."encrypted_url_variables", "web_hooks"."encrypted_url_variables_iv", "web_hooks"."integration_id", "web_hooks"."branch_filter_strategy", "web_hooks"."emoji_events", "web_hooks"."name", "web_hooks"."description", "web_hooks"."custom_webhook_template", "web_hooks"."resource_access_token_events", "web_hooks"."encrypted_custom_headers", "web_hooks"."encrypted_custom_headers_iv", "web_hooks"."project_events", "web_hooks"."vulnerability_events", "web_hooks"."member_approval_events", "web_hooks"."milestone_events", "web_hooks"."organization_id", "web_hooks"."resource_deploy_token_events", "web_hooks"."filter", "web_hooks"."signing_token", "web_hooks"."duo_flow_callback_enabled" FROM "web_hooks" WHERE "web_hooks"."type" = 'GroupHook' AND "web_hooks"."group_id" IN (SELECT "namespaces"."id" FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" IN (9970, 2452873, 2564205)))) web_hooks WHERE "web_hooks"."duo_flow_callback_enabled" = TRUE

Postgres AI Session link

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited Aug 26, 2026 by Thomas Schmidt

Wire callback_hook_id into the create-flow API