Remove the organization-scoped Duo flow callback registration API

Thomas Schmidtrequested to merge
duo/fix/627862-remove-org-flow-callbacks-api into master

What does this MR do and why?

Removes the organization-scoped Duo flow callback registration API — the four endpoints under /ai/duo_workflows/flow_callbacks added in !247573 (merged) (GitLab 19.3) — together with its model, entity, permissions, and specs, and deletes the rows it created.

Why this API has to go: master has two ways to register a Duo flow callback endpoint, and only one of them works. This one shipped registration-only in 19.3; its delivery follow-up was superseded by gitlab-org#22922 (closed). The organization-scoped model was never wired up:

  • API::Ai::DuoWorkflows::Workflows#build_messaging_callback_context resolves callback_hook_id only through Ai::DuoWorkflows::CallbackHooksFinder.
  • That finder returns project and ancestor-group hooks with duo_flow_callback_enabled: true only.
  • The flow_callbacks create path never sets that column, and a FlowCallbackHook is organization-scoped, so it can never fall inside the finder's scope.

Net effect: an ID returned by this API is rejected with 400 at trigger time. No delivery path ever existed, so no integration can be receiving — or ever have received — a callback through it.

What changes:

  • Deletes API::Ai::DuoWorkflows::FlowCallbacks, Ai::DuoWorkflows::FlowCallbackHook, the Grape entity, the request/model specs, the factory, and the duo_flow_callback_hook permission + assignable-permission definitions.
  • Drops the API mount, the three *_duo_flow_callback_hook abilities from the organization policy, the db/docs/web_hooks.yml class entry, and the gitlab.pot msgid.
  • Regenerates config/routing/gitlab_routes.json, doc/api/openapi/openapi_v3.yaml, and doc/auth/tokens/fine_grained_access_tokens_rest.md.
  • Removes the four "flow callback endpoint" sections from doc/api/duo_agent_platform_flows.md. The same deletion landed in !253875 (merged), which has since merged.
  • Adds post-deploy migration 20260908120000_remove_duo_flow_callback_hooks_from_web_hooks.rb, which deletes web_hooks rows with type = 'Ai::DuoWorkflows::FlowCallbackHook' in batches and, defensively, their web_hook_logs_daily rows. Log rows should not exist: only delivery writes them, and delivery never existed. Deletion over a type rewrite because the rows are inert registrations with no state worth keeping, and no other STI class fits the organization scope. A leftover row otherwise raises ActiveRecord::SubclassNotFound once the class is gone (reproduced on a GDK).

Deprecation: no data/deprecations/ entry. The endpoints could never perform their documented purpose, so no customer workflow can be disrupted; the docs style guide also permits deleting content for not-generally-available features outright (applied in the docs MR). The issue's acceptance criterion was adjusted accordingly; if a reviewer wants the entry anyway, it is a small follow-up.

Working path untouched: duo_flow_callback_enabled, CallbackHooksFinder, Ai::Messaging::Adapters::Webhook, Ai::Messaging::WebhookDeliveryWorker, and the duo_flow_callback_hooks feature flag are all unchanged.

Database review

Records affected on GitLab.com: 0.

An unbounded scan for the type returns no rows on a production clone, so the each_batch loop body never runs and neither DELETE executes. Nobody ever registered a hook through this API, which is consistent with the returned IDs being unusable at trigger time.

Plans captured on gitlab-production-main (postgres.ai session 57539, clone state 2026-09-16):

Query Plan Result
Full row count for the type 162106 rows=0, 3 buffers, 0.03 ms
each_batch boundary scan 162099 rows=0, 6 buffers, 0.05 ms
DELETE from web_hook_logs_daily 162100 rows=0, 3 buffers, 0.64 ms
DELETE from web_hooks 162102 rows=0, 3 buffers, 0.07 ms 
-- 0. full row count for the type (no LIMIT)
SELECT id FROM web_hooks
 WHERE type = 'Ai::DuoWorkflows::FlowCallbackHook';

-- 1. each_batch boundary scan
SELECT id FROM web_hooks
 WHERE type = 'Ai::DuoWorkflows::FlowCallbackHook'
 ORDER BY id ASC LIMIT 1;

-- 2. per-batch log deletion
DELETE FROM web_hook_logs_daily
 WHERE web_hook_id IN (
   SELECT id FROM web_hooks
    WHERE type = 'Ai::DuoWorkflows::FlowCallbackHook'
      AND id >= 1 AND id < 1001);

-- 3. per-batch hook deletion
DELETE FROM web_hooks
 WHERE type = 'Ai::DuoWorkflows::FlowCallbackHook'
   AND id >= 1 AND id < 1001;

The migration testing pipeline confirms this end to end: it ran in 3.0 s with a +0.00 B database size change, and the only statement it issued was the each_batch boundary scan returning 0 rows. Neither DELETE appears in the query list, because the loop body never ran.

Both deletes are index-backed: index_web_hooks_on_type on web_hooks, and index_web_hook_logs_daily_on_web_hook_id_and_created_at per partition.

Why the web_hook_logs_daily deletion stays. The table is partitioned by created_at and the delete filters only on web_hook_id, so partition fan-out is a fair question. The plan shows it does not matter: the web_hooks index scan is the outer side of the nested loop and returns zero rows, so the inner partition probes execute zero times (3 buffers total). The only fan-out cost is planning — 12.4 ms against 0.9 ms for the non-partitioned delete — paid once per batch, and there are no batches. The statement is kept for self-managed instances, where the per-partition index handles any rows that exist.

Recovery. down is a deliberate no-op. The rows are inert registrations with no state, no delivery path ever read them, and GitLab.com has none to lose. A self-managed instance can re-register once the replacement API in gitlab-org#22922 (closed) lands.

User experience impact. None. The endpoints returned callback hook IDs that API::Ai::DuoWorkflows::Workflows rejects with 400, so no working integration exists to break.

References

Screenshots or screen recordings

No user-visible UI change; the removal is API, backend, and documentation only.

Before After
n/a n/a

How to set up and validate locally

I think there is not really something to validate locally here. We could make sure that old functionality still works but this is mostly deleting dead code.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Thomas Schmidt

Merge request reports