Draft: Parse only notes with user references when collecting participants

For every new note, NewNoteWorker builds notification recipients from the participants of the whole issue or MR, and that walk parsed the HTML of every note to find user mentions. A note's cached HTML marks every user reference with data-reference-type="user", so notes without that attribute can skip the parse without dropping anyone.

This is a partial fix. Locally, on an issue with 1,000 notes, collecting participants took about 25% less CPU (median 294 ms instead of 391 ms), with fewer GC spikes. It does not remove the O(N) walk: every note is still loaded and its markdown cache checked. It is behind the participants_parse_only_notes_with_user_references flag, which is off by default.

Related to https://gitlab.com/gitlab-org/gitlab/-/work_items/634351

Detailed context for AI agents

Background

INC-15305 (S2, 2026-10-16): a bot replied about 9.7 times per second into one MR discussion (about 5,000 notes on the MR). https://gitlab.com/gitlab-org/gitlab/-/work_items/634351 shows NewNoteWorker median CPU rising from 0.44 s to 3.95 s as the MR grew, with a flat query count, and the worker becoming the second-largest group of LWLock:LockManager waiters on the primary.

Call path: NewNoteWorker#perform -> NotificationService#new_note -> NotificationRecipients::Builder::NewNote#build! -> add_participants(note.author) -> noteable.participants(user) -> Participable#raw_participants. For an issuable this walks notes_for_participants (every user note plus system notes that may mention someone, up to 10,000). For each note it runs the participant attrs Note.participant_attrs = [Mentionable lambda, :award_emoji, :author]. The lambda calls all_references(user, extractor: ext), which queues the note's text and cached HTML on the shared extractor. At the end, extractor.users runs Banzai::Renderer.cache_collection_render and then Nokogiri::HTML5.fragment on every queued document, and UserParser looks for a[data-reference-type='user'].gfm nodes.

Change

In Participable#raw_participants, with the flag on, the proc attrs are skipped for a Note source whose updated_cached_html_for(:note) is present and does not match /data-reference-type=["']user["']/. The symbol attrs (:author, :award_emoji) still run, so note authors and award emoji users are unchanged.

Why no participant is dropped:

  • raw_participants only reads extractor.users and internal_notes_extractor.users (via users_that_can_read_internal_notes). No other reference types from these extractors are used.
  • UserParser only finds nodes with data-reference-type='user', which covers user, group and @all or project mentions (all rendered by UserReferenceFilter with that type). A document without the attribute contributes no nodes.
  • The HTML checked is the same string the extractor would parse: Mentionable#all_references passes updated_cached_html_for(attr) as rendered, and cache_collection_render uses rendered as-is.
  • updated_cached_html_for re-renders stale caches (markdown version bump) before returning, so a stale cache is checked after refresh, exactly as the extractor would see it. Blank HTML is not skipped.
  • The check sits after skippable_system_notes?, so system notes behave as before.

One case returns more users with the flag on, not fewer. This is a bug on master, not something this MR adds:

  • When the current user wrote a note whose markdown cache is stale or empty, the Mentionable lambda sets extractors[current_user] to the shared extractor, then updated_cached_html_for refreshes the cache. refresh_markdown_cache! -> mentions_to_store -> all_references(author) finds that same extractor under the author and calls reset_memoized_values, which drops every text queued so far, for example the mentions in the issue description.
  • With the flag on, the check calls updated_cached_html_for before the lambda has registered the shared extractor, so the refresh builds its own extractor and the queued texts survive.
  • Reproduced locally with a read-only script, and pinned by a spec (below) that fails with the flag off.

The flag is checked once per raw_participants call with extractor.project as the actor. That is the same value as project, but it avoids an extra project call, which existing specs count exactly.

Measurement (local, not production)

Throwaway spec: a public project issue with 1,000 user notes (a paragraph, inline code, bold, a code block and a list each, with fresh caches) plus one note mentioning a user. It measures process CPU time of Issue.find(id).participants(user), alternating flag on and off after warmup:

  • flag on: 252, 271, 279, 283, 294, 299, 309, 313, 414 ms (median 294)
  • flag off: 325, 366, 367, 384, 386, 391, 408, 661, 738, 750, 781 ms (median 391; the four slowest look like GC pauses)
  • The first run of each batch (both flag on, 502 and 527 ms) was warmup and is left out.
  • notes_for_participants.to_a alone: 64 to 106 ms. updated_cached_html_for(:note) over all 1,000 notes: 115 to 157 ms. The old path pays both as well.

So this removes the parse, but loading every note and checking its markdown cache is still O(N) per new note.

Further options, not in this MR

  • Load only notes whose note_html may contain a user reference (or whose cache is stale) and pluck author and award emoji user ids for the rest. That moves most of the walk into SQL, but changes Issuable#notes_for_participants and must keep the 10,000-note limit semantics.
  • Thread-only participants for new-note notifications, or incrementally cached participants. Both change who gets "participating" notifications, so they need a product decision.
  • Relaxing NewNoteWorker's data_consistency :always so its reads can use replicas.

Feature flag

  • participants_parse_only_notes_with_user_references, gitlab_com_derisk, group::code review, milestone 19.6, actor project (nil for group-level and personal snippet participables, which switch over only when the flag is fully on).
  • Off: unchanged behaviour.
  • Rollout issue: #634764
  • Also affects the other callers of the same walk: NotificationRecipients::Builder::Default (state-change notifications), Users::ParticipableService#participants_in_noteable (mention autocomplete) and participant?.

Tests

spec/models/concerns/participable_spec.rb, new context "with notes that have no user references". The issue has a plain note, a note referencing an issue, a note mentioning a user, and an internal note mentioning a reporter:

  • flag on and off: participants are the issue author, the note author, the mentioned user and the reporter
  • flag on: only the two notes with user mentions are passed to cache_collection_render
  • flag off: every note is
  • flag on: a note mentioning a public group still adds the group's members
  • flag on: when the current user wrote a note with an empty note_html, a user mentioned only in the issue description is still a participant (the master bug above)

Mentioned users are created :with_namespace, because factory users have no namespace by default and UserReferenceFilter only links users through their namespace.

Results (local, GDK): participable_spec 33/0, projects/participants_service_spec 29/0, EE projects/participants_service_spec 6/0, notification_recipients/build_service_spec 6/0, notification_recipients/builder/new_note_spec 2/0, notification_recipients/builder/default_spec 23/0, note_spec 345/0. A local break matrix (15 scenarios, not in the MR) compared flag on and off participants across note types. RuboCop clean. Mutation check: a pattern that never matches makes the participant examples fail, because mentions get skipped.

Edited by Marc Shaw

Merge request reports

Loading
Loading