Dispatch APNs push notifications on todo creation

Dispatch APNs push notifications on todo creation

What does this MR do and why?

Completes the mobile push notifications stack: every newly created to-do for a user with registered devices produces an APNs push, behind two feature flags (both disabled by default): mobile_push_notifications_dispatch gates enqueues instance-wide, mobile_push_notifications gates delivery per recipient user.

Part 4 of 4 (stacked on the registration API MR):

1. apnotic gem — !248021 (merged)
2. Device push subscription registry — !248022 (merged)
3. Registration REST API — !248023 (merged)
4. → this MR: to-do dispatch worker + APNs delivery

How it works

• TodoService#create_todos enqueues one batched Todos::PushNotificationWorker.perform_async(todo_ids) per call, deferred via Todo.current_transaction.after_commit so it never races the bulk insert's transaction. The enqueue is double-gated: the instance-wide mobile_push_notifications_dispatch flag keeps the todo path free of any reads or enqueues while the feature is dark (rolling-deploy safety + kill switch), and a subscription pre-filter (Notifications::MobileDevicePushSubscription.subscribed_user_ids, one indexed query per batch, after commit) drops recipients without a registered device, so a batch whose recipients have no devices enqueues nothing.
• The worker is a thin shell around Notifications::MobilePush::SendTodoNotificationsService, which loads the batch with preloaded users and subscriptions and enforces the mobile_push_notifications flag per recipient; every skip reason is a result label in the metrics, and the worker logs the returned tallies.
• The service skips resolved to-dos (mark-done races) and inactive users, builds the alert payload from the to-do (title/subtitle/body mirror the mobile app's to-do copy, badge = cached pending count, per-todo apns-collapse-id, typed routing keys + target_url in a custom dict), and sends over an HTTP/2 connection that is opened once per APNs environment and reused across the batch — sandbox or production host per subscription.
• payload_mode: id_only subscriptions get content-free payloads (generic copy, {version, type, todo_id, user_id} only).
• Dead tokens (Unregistered / BadDeviceToken) evict the subscription.
• Per-user rate limiting via Gitlab::ApplicationRateLimiter (mobile_push_notifications): on breach, one generic summary alert per window, further alerts suppressed.
• Notifications::MobilePush::PruneStaleSubscriptionsWorker (daily cron) deletes subscriptions unseen for 90 days.
• Metrics: mobile_push_notifications_total{result} counter + creation-to-delivery latency histogram; structured worker logs carry result/subscription count and never include alert content.

Configuration

APNs credentials come from gitlab.yml (mobile_push.apns: auth_key_path, key_id, team_id, optional topic), wired with defaults in 1_settings.rb like the other instance settings; without a key the worker logs and no-ops, so instances without keys are unaffected.

How to test

With the stack checked out, flags enabled, mobile_push.apns configured in gitlab.yml, and a device registered (parts 1–3): assign an issue to yourself from another user — a push arrives on the registered device. Without credentials: Todos::PushNotificationWorker.perform_async(Todo.pending.last.id) logs skipped_not_configured.

Feature flag rollout

• mobile_push_notifications_dispatch — instance-wide enqueue gate and kill switch, deliberately actor-less; enabled globally (step 1) once the worker class is deployed everywhere.
• mobile_push_notifications, actor: user — per-recipient delivery, enforced in the worker; rolled out per user / percentage of actors (step 2).
• Rollout issue for both: #607603.

🤖 Generated with Claude Code

Database

No schema changes — this MR only reads from and deletes rows of the mobile_device_push_subscriptions table introduced in !248022 (merged). The table now exists on GitLab.com (!248022 (merged) merged), but it is still empty — devices cannot register until the registration API (!248023 (merged)) ships — so there is no production data for postgres.ai to clone. The plans below are EXPLAIN (ANALYZE, BUFFERS) from a local PostgreSQL 17.8 instance running the merged schema (all three indexes), seeded with 100,000 rows across 5,000 users — 20 subscriptions per user, i.e. every user at the per-user cap — with last_seen_at spread uniformly over the past 180 days (≈50 % of the table stale at the 90-day cutoff, the worst case for the pruner). ANALYZE was run after seeding.

TodoService#create_todos dispatch pre-filter (request path)

0. Subscribed-recipient check — Notifications::MobileDevicePushSubscription.subscribed_user_ids(user_ids), one query per todo-creation batch, issued in the after-commit continuation behind the instance-wide mobile_push_notifications_dispatch flag (feature dark ⇒ zero queries on the todo path):

SELECT DISTINCT "mobile_device_push_subscriptions"."user_id" FROM "mobile_device_push_subscriptions" WHERE "mobile_device_push_subscriptions"."user_id" IN (101, 102, 103) LIMIT 3

Same access path as the subscriptions preload in plan 2 (index_mobile_device_push_subscriptions_on_user_id, IN list bounded by the batch's distinct recipients); it reads only user_id, so PostgreSQL can serve it index-only, and the LIMIT (the number of distinct ids passed in) makes the result bound explicit.

Todos::PushNotificationWorker (one job per TodoService call)

1. Todo load — Todo.id_in(todo_ids).pending.with_preloaded_user_and_push_subscriptions, where todo_ids is the pre-filtered id list of a single todo-creation batch (only todos whose recipient has a subscription):

SELECT "todos".* FROM "todos" WHERE "todos"."id" IN (27, 28, 29) AND "todos"."state" = 'pending'

Execution plan

Index Scan using index_on_todos_user_project_target_and_state on todos  (cost=0.14..2.17 rows=1 width=190) (actual time=0.166..0.168 rows=3 loops=1)
  Index Cond: (id = ANY ('{27,28,29}'::bigint[]))
  Buffers: shared hit=1 read=1
Planning Time: 2.597 ms
Execution Time: 0.182 ms

(The todos table is development-scale here, hence the planner's index choice; the query itself is a batch-bounded id lookup. The preloads for author/note/target/project/group routes are the standard todos-domain association lookups that every todo consumer issues.)

2. Subscriptions preload (user: :mobile_device_push_subscriptions) — one IN query over the distinct user ids of the loaded todos; row count is bounded by users × 20 (the per-user cap enforced in !248022 (merged)):

SELECT "mobile_device_push_subscriptions".* FROM "mobile_device_push_subscriptions" WHERE "mobile_device_push_subscriptions"."user_id" IN (101, 102, 103)

Execution plan

Index Scan using index_mobile_device_push_subscriptions_on_user_id on mobile_device_push_subscriptions  (cost=0.29..64.79 rows=60 width=277) (actual time=0.034..0.220 rows=60 loops=1)
  Index Cond: (user_id = ANY ('{101,102,103}'::bigint[]))
  Buffers: shared hit=62
Planning Time: 5.462 ms
Execution Time: 0.272 ms

3. Bad-token cleanup — subscription.destroy when APNs reports the token unregistered (HTTP 410 / BadDeviceToken); a primary-key delete of an already-loaded record, issued at most once per subscription per delivery:

DELETE FROM "mobile_device_push_subscriptions" WHERE "mobile_device_push_subscriptions"."id" = 55000

Execution plan

Delete on mobile_device_push_subscriptions  (cost=0.29..2.31 rows=0 width=0) (actual time=0.181..0.181 rows=0 loops=1)
  Buffers: shared hit=8
  ->  Index Scan using mobile_device_push_subscriptions_pkey on mobile_device_push_subscriptions  (cost=0.29..2.31 rows=1 width=6) (actual time=0.034..0.034 rows=1 loops=1)
        Index Cond: (id = 55000)
        Buffers: shared hit=6
Planning Time: 0.078 ms
Execution Time: 0.484 ms

Notifications::MobilePush::PruneStaleSubscriptionsWorker (cron)

Notifications::MobileDevicePushSubscription.stale(90.days.ago).each_batch(of: 1000) { |batch| batch.delete_all }. The worker declares defer_on_database_health_signal :gitlab_main, [:mobile_device_push_subscriptions], 5.minutes, so it backs off under database pressure. each_batch walks the primary key, so each iteration is three statements:

4. First batch lower bound:

SELECT "mobile_device_push_subscriptions"."id" FROM "mobile_device_push_subscriptions" WHERE "mobile_device_push_subscriptions"."last_seen_at" < '2026-05-07 15:30:00' ORDER BY "mobile_device_push_subscriptions"."id" ASC LIMIT 1

Execution plan

Limit  (cost=0.29..0.41 rows=1 width=8) (actual time=0.020..0.020 rows=1 loops=1)
  Buffers: shared hit=3
  ->  Index Scan using mobile_device_push_subscriptions_pkey on mobile_device_push_subscriptions  (cost=0.29..5873.29 rows=50015 width=8) (actual time=0.019..0.019 rows=1 loops=1)
        Filter: (last_seen_at < '2026-05-07 15:30:00+00'::timestamp with time zone)
        Buffers: shared hit=3
Planning Time: 0.143 ms
Execution Time: 0.028 ms

5. Next-batch upper bound (repeated per batch with a moving id >= cursor):

SELECT "mobile_device_push_subscriptions"."id" FROM "mobile_device_push_subscriptions" WHERE "mobile_device_push_subscriptions"."last_seen_at" < '2026-05-07 15:30:00' AND "mobile_device_push_subscriptions"."id" >= 1 ORDER BY "mobile_device_push_subscriptions"."id" ASC LIMIT 1 OFFSET 1000

Execution plan

Limit  (cost=122.72..122.84 rows=1 width=8) (actual time=1.114..1.115 rows=1 loops=1)
  Buffers: shared hit=81
  ->  Index Scan using mobile_device_push_subscriptions_pkey on mobile_device_push_subscriptions  (cost=0.29..6123.29 rows=50015 width=8) (actual time=0.017..1.049 rows=1001 loops=1)
        Index Cond: (id >= 1)
        Filter: (last_seen_at < '2026-05-07 15:30:00+00'::timestamp with time zone)
        Rows Removed by Filter: 909
        Buffers: shared hit=81
Planning Time: 0.088 ms
Execution Time: 1.122 ms

6. Batch delete (id-range bounded, ≤ 1,000 rows per statement):

DELETE FROM "mobile_device_push_subscriptions" WHERE "mobile_device_push_subscriptions"."last_seen_at" < '2026-05-07 15:30:00' AND "mobile_device_push_subscriptions"."id" >= 1 AND "mobile_device_push_subscriptions"."id" < 1910

Execution plan

Delete on mobile_device_push_subscriptions  (cost=0.29..126.28 rows=0 width=0) (actual time=4.248..4.249 rows=0 loops=1)
  Buffers: shared hit=1155
  ->  Index Scan using mobile_device_push_subscriptions_pkey on mobile_device_push_subscriptions  (cost=0.29..126.28 rows=978 width=6) (actual time=0.024..1.236 rows=1000 loops=1)
        Index Cond: ((id >= 1) AND (id < 1910))
        Filter: (last_seen_at < '2026-05-07 15:30:00+00'::timestamp with time zone)
        Rows Removed by Filter: 909
        Buffers: shared hit=81
Planning Time: 0.071 ms
Execution Time: 4.261 ms

At the seeded 50 % staleness the planner walks the primary key and filters on last_seen_at (909 non-stale rows skipped per bound query above); with production-realistic sparse staleness the index_mobile_device_push_subscriptions_on_last_seen_at index added in !248022 (merged) serves the scope directly. Either way every statement stays bounded by the 1,000-row batch and the moving id cursor.