Suggested changes for the subscription coalescer

Suggested changes for !266624 (merged).

This MR targets your branch 630854-subscription-coalescer, so you can merge it there. The commits carry my name. Rebase and reset the author if you want them under yours.

Two commits fix bugs:

  1. A key can stop flushing. If the scheduler raises in Coalescer#submit, the entry stays and no flush is scheduled. Later updates for that key are never sent.
  2. Old updates carry into the next window. entry.latest is not reset after a flush, and one blank WAL location then pins every later merged update for that key to the primary.

The other commits move code out of ActionCableWithLoadBalancing and follow existing patterns. Each commit message says "Behavior change" or "Structure only", so you can review the structure commits quickly.

Commit Type
Use Concurrent::Map in the subscription coalescer behavior
Extract Envelope from ActionCableWithLoadBalancing structure
Compare WAL locations with an Lsn value object behavior
Add CoalescePolicy for subscriptions that coalesce structure
Resolve the coalesce policy once for each stream performance
Extract subscribers_of and rename defer_update structure
Inject the subscriptions into ActionCableSampler structure
How I checked it
  • 21 fast examples and 228 integration examples pass, and RuboCop reports no offenses.
  • A deferred submit takes 1.26 µs, against 0.38 µs for the Mutex version. With the real WAL merge it takes 5.05 µs, against 2.16 µs. A deferred update replaces a full subscription run that takes milliseconds.
  • Before the "once for each stream" commit, every message on every subscriber stream paid about 1.6 µs for a schema lookup. After it, subscriptions without coalesce pay nothing.
  • The cap check and the topic counter are separate steps, so the key caps can go over by about the size of the worker pool.
  • The block given to @bursts.compute must not call @bursts again, because the map's lock cannot be taken twice by one thread.

Merge request reports

Loading
Loading