Add backfill worker for DuoChatEvents

Pavel Shutsinrequested to merge
523406-introduce-backfill-worker-for-duo_chat_events-clickhouse-table into master

What does this MR do and why?

This worker will be used to copy data from PG to ClickHouse when instance admin enabled use_clickhouse_for_analytics setting. In scope of this MR we only introduce worker functionality without attaching it to application setting. No user facing changes here.

How to set up and validate locally

  1. Enable application setting Gitlab::CurrentSettings.current_application_settings.update(use_clickhouse_for_analytics: true)
  2. Seed ai_usage_stats for given project PROJECT_ID=2 FILTER=ai_usage_stats bundle exec rake db:seed_fu
  3. In Rails console wipe CH usage table: ClickHouse::Client.execute("ALTER TABLE duo_chat_events DELETE WHERE 1=1", :main)
  4. In Rails console run the worker: Analytics::DuoChatEventsBackfillWorker.new.handle_event(nil)
  5. Verify that data is present in CH: ClickHouse::Client.select("SELECT * FROM duo_chat_events WHERE 1=1", :main)

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #523406

Edited by Pavel Shutsin

Merge request reports