Skip to content

Add no-op worker for syncing events to ClickHouse

Adam Hegyi requested to merge 414937-implement-no-op-periodical-ch-data-sync-worker into master

What does this MR do and why?

This PR implements a currently no-op worker for syncing data from the events table to ClickHouse. This first MR configures the worker:

  • Lock to avoid parallel execution.
  • Schedule the worker only on SaaS, every 3 minutes.
  • Log metadata and extra stats.

Why is the worker no-op? The batching and syncing logic is a bit complex and I couldn't find a way to split it down to meaningful chunks. So I decided to add the worker and its configuration first.

How to set up and validate locally

  1. Configure a ClickHouse database
  cp config/click_house.yml.example config/click_house.yml
  # fill in the development environment (does not require running service)
  1. Enable the FF: Feature.enabled?(:event_sync_worker_for_click_house)
  2. Invoke: ClickHouse::EventsSyncWorker.new.perform

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #414937 (closed)

Edited by Adam Hegyi

Merge request reports