Skip to content

Worker for ingesting code suggestion events

Adam Hegyi requested to merge 441485-worker-for-ingesting-cs-data into master

What does this MR do and why?

This MR ingests code suggestion usage data to ClickHouse from a redis buffer. The change is behind a feature flag: code_suggestion_events_in_click_house

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

How to set up and validate locally

  1. Ensure that ClickHouse is set up for your GDK: https://docs.gitlab.com/ee/development/database/clickhouse/clickhouse_within_gitlab.html
  2. In rails console enable following settings 
    Feature.enable(:code_suggestion_events_in_click_house)
Feature.enable(:ai_tracking_data_gathering)
Gitlab::CurrentSettings.current_application_settings.update(use_clickhouse_for_analytics: true)
  3. Put some data in the buffer: 
      10000.times { Gitlab::Tracking::AiTracking.track_event('code_suggestions_requested', { user_id: rand(100) }) }
  4. Invoke the worker 
      ClickHouse::CodeSuggestionEventsCronWorker.new.perform
  5. Verify that data is in clickhouse: 
    ClickHouse::Client.select('select count(*) from code_suggestion_usages final', :main)

Related to #441485 (closed)

Edited by Adam Hegyi

Merge request reports