Skip to content

Add publish_group feature to Gitlab::EventStore

Sashi Kumar Kumaresan requested to merge sk/424938-group-publish into master

What does this MR do and why?

This MR adds functionality to publish group of events through the GitLab EventStore. Currently, when an event is published to EventStore, a worker job is invoked for the event. There are a few cases where we dispatch a lot of events of same type in a single business transaction. One example is, when a user is added to a group, we will dispatch ProjectAuthorizations::AuthorizationsChangedEvent for each of the projects within the group. This results in a number of sidekiq workers invoked at the same time thereby putting pressure on sidekiq.

With publish_group, we can dispatch a group of events of same type so that the subscriber worker receives a batch of events instead of single event. The group size could be configured using group_size parameter while creating subscription.

Addresses #424938 (closed)

Example

Subscription

store.subscribe ::Security::RefreshProjectPoliciesWorker, to: ::ProjectAuthorizations::AuthorizationsChangedEvent, group_size: 25

Publishing events

events = project_ids.map do |project_id|
  ProjectAuthorizations::AuthorizationsChangedEvent.new(data: { project_id: project_id })
end
Gitlab::EventStore.publish_group(events)

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Sashi Kumar Kumaresan

Merge request reports