Preload only relevant notification settings for recipients
What does this MR do and why?
NotificationRecipients::Builder::Base loads recipients that come from a relation (project and group watchers, custom notification levels, subscribers and assignees) with includes(:notification_settings), which loads every notification setting each recipient has. filter! then runs a scoped preload for only the event's project, its group hierarchy and the global setting, but Rails skips associations that are already loaded, so the scoped preload never applies to these recipients. The cost of a notification job grows with each recipient's total number of settings, not with the settings that apply to the event. Users who belong to many projects or groups pay it on every event that reaches them.
Behind the notification_recipients_scoped_settings_preload feature flag (gitlab_com_derisk, disabled by default, project actor):
add_recipientsanduser_scopeno longer add the unscopedincludes, so the existing scoped preload infilter!loads the settings for all recipients in one query.- After filtering, the association is reset for every candidate, including filtered-out ones, so callers never see a partial association. Some of these
Userobjects can belong to the caller, for example a target's assignees.
This follows the same pattern as 8c169c1b in BuildService.notifiable_users. Every lookup in NotificationRecipient#notifiable? (project setting, closest group setting, global setting and the group notification email) reads only settings inside the scoped preload, so the recipients and reasons are the same with the flag on and off.
Group-level targets have no project, so they follow the flag's global state only, not a percentage rollout.
No changelog entry while the flag is disabled by default.
Local measurement
One watcher with 50,000 settings for other sources, one NotificationRecipients::Builder::Default build for a new issue (test database):
| Settings rows loaded | CPU | |
|---|---|---|
| Flag disabled | 50,001 | 2.16 s |
| Flag enabled | 2 | 0.015 s |
Database
No new or modified queries. These recipients no longer run the unscoped preload:
SELECT "notification_settings".* FROM "notification_settings" WHERE "notification_settings"."user_id" IN (...)They're covered by the scoped preload that filter! already runs for every build. Both queries are served by the unique index index_notifications_on_user_id_and_source_id_and_source_type (user_id, source_id, source_type), and the scoped query reads only the rows for the event's sources:
SELECT "notification_settings".* FROM "notification_settings"
WHERE ("notification_settings"."source_type" = 'Project' AND "notification_settings"."source_id" = ...
OR "notification_settings"."source_type" = 'Namespace' AND "notification_settings"."source_id" IN (...)
OR "notification_settings"."source_id" IS NULL)
AND "notification_settings"."user_id" IN (...)References
- https://gitlab.com/gitlab-org/gitlab/-/work_items/635895
- Rollout issue: #639143
- Same pattern: 8c169c1b
Screenshots or screen recordings
Not applicable, no UI change.
How to set up and validate locally
- In the Rails console, give a user who watches a project settings for many other sources:
user = User.find_by_username('<username>') project = Project.find_by_full_path('<project path>') NotificationSetting.find_or_create_by!(user: user, source: project).update!(level: :watch) now = Time.current NotificationSetting.insert_all((1..20_000).map { |i| { user_id: user.id, source_type: 'Project', source_id: -i, level: 0, created_at: now, updated_at: now } }) - Time a recipient build for an issue in that project, with the flag off and on:
Both builds return the same recipients. With the flag enabled, the build doesn't load the 20,000 extra rows.
issue = project.issues.last build = -> { NotificationRecipients::Builder::Default.new(issue, issue.author, action: :new).notification_recipients } Feature.disable(:notification_recipients_scoped_settings_preload) Benchmark.realtime { build.call } Feature.enable(:notification_recipients_scoped_settings_preload) Benchmark.realtime { build.call } - Clean up with
NotificationSetting.where(user: user).where('source_id < 0').delete_all.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.