Merge branch 'ag-add-cs-usage-fixtures' into 'master'

Add fixtures for code suggestion usage

See merge request !149042



Merged-by: Pavel Shutsin <pshutsin@gitlab.com>
Approved-by: Pavel Shutsin <pshutsin@gitlab.com>
Co-authored-by: Adam Hegyi <ahegyi@gitlab.com>

doc/development/development_seed_files.md

@@ -14,14 +14,15 @@ The task `rake db:seed_fu` can be used to run all development seeds with the exc
 The following table summarizes the seeds and tasks that can be used to generate
 data for features.
 
-| Feature                                                                                                           | Command                                                                                                           | Seed                                                                                                                                                |
+| Feature                                                                                                           | Command                                                                                                       | Seed                                                                                                                                                |
 |-------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------|
 | DevOps Adoption                                                                                                   | `FILTER=devops_adoption bundle exec rake db:seed_fu`                                                          | [31_devops_adoption.rb](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/db/fixtures/development/31_devops_adoption.rb)                        |
 | Value Streams Dashboard                                                                                           | `FILTER=cycle_analytics SEED_VSA=1 bundle exec rake db:seed_fu`                                               | [17_cycle_analytics.rb](https://gitlab.com/gitlab-org/gitlab/-/blob/master/db/fixtures/development/17_cycle_analytics.rb)                           |
 | Value Streams Dashboard overview counts                                                                           | `FILTER=vsd_overview_counts SEED_VSD_COUNTS=1 bundle exec rake db:seed_fu`                                    | [93_vsd_overview_counts.rb](https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/db/fixtures/development/93_vsd_overview_counts.rb)                |
-| Value Stream Analytics                                                                                            | `FILTER=customizable_cycle_analytics SEED_CUSTOMIZABLE_CYCLE_ANALYTICS=1 bundle exec rake db:seed_fu` | [30_customizable_cycle_analytics](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/db/fixtures/development/30_customizable_cycle_analytics.rb) |
+| Value Stream Analytics                                                                                            | `FILTER=customizable_cycle_analytics SEED_CUSTOMIZABLE_CYCLE_ANALYTICS=1 bundle exec rake db:seed_fu`         | [30_customizable_cycle_analytics](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/db/fixtures/development/30_customizable_cycle_analytics.rb) |
 | CI/CD analytics                                                                                                   | `FILTER=ci_cd_analytics SEED_CI_CD_ANALYTICS=1 bundle exec rake db:seed_fu`                                   | [38_ci_cd_analytics](https://gitlab.com/gitlab-org/gitlab/-/blob/master/db/fixtures/development/38_ci_cd_analytics.rb?ref_type=heads)               |
-| Contributions Analytics<br><br>Productivity Analytics<br><br>Code review Analytics<br><br>Merge Request Analytics | `FILTER=productivity_analytics SEED_PRODUCTIVITY_ANALYTICS=1 bundle exec rake db:seed_fu`             | [90_productivity_analytics](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/db/fixtures/development/90_productivity_analytics.rb)             |
-| Repository Analytics                                                                                              | `FILTER=14_pipelines NEW_PROJECT=1 bundle exec rake db:seed_fu`                                       | [14_pipelines](https://gitlab.com/gitlab-org/gitlab/-/blob/master/db/fixtures/development/14_pipelines.rb?ref_type=heads)                           |
+| Contributions Analytics<br><br>Productivity Analytics<br><br>Code review Analytics<br><br>Merge Request Analytics | `FILTER=productivity_analytics SEED_PRODUCTIVITY_ANALYTICS=1 bundle exec rake db:seed_fu`                     | [90_productivity_analytics](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/db/fixtures/development/90_productivity_analytics.rb)             |
+| Repository Analytics                                                                                              | `FILTER=14_pipelines NEW_PROJECT=1 bundle exec rake db:seed_fu`                                               | [14_pipelines](https://gitlab.com/gitlab-org/gitlab/-/blob/master/db/fixtures/development/14_pipelines.rb?ref_type=heads)                           |
 | Issue Analytics<br><br>Insights                                                                                   | `NEW_PROJECT=1 bin/rake gitlab:seed:insights:issues`                                                          | [insights Rake task](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/lib/tasks/gitlab/seed/insights.rake)                                     |
 | DORA metrics                                                                                                      | `SEED_DORA=1 FILTER=dora_metrics bundle exec rake db:seed_fu`                                                 | [92_dora_metrics](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/db/fixtures/development/92_dora_metrics.rb)                                 |
+| Code Suggestion data in ClickHouse                                                                                | `FILTER=ai_usage_stats bundle exec rake db:seed_fu`                                                           | [94_ai_usage_stats](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/db/fixtures/development/94_ai_usage_stats.rb)                             |

ee/db/fixtures/development/94_ai_usage_stats.rb

+# frozen_string_literal: true
+
+# Usage:
+#
+# Seeds all groups:
+#
+# FILTER=ai_usage_stats bundle exec rake db:seed_fu
+#
+# Invoking for a single project:
+#
+# PROJECT_ID=22 FILTER=ai_usage_stats bundle exec rake db:seed_fu
+
+# rubocop:disable Rails/Output -- this is a seed script
+class Gitlab::Seeder::AiUsageStats # rubocop:disable Style/ClassAndModuleChildren -- this is a seed script
+  CODE_PUSH_SAMPLE = 10
+  AI_EVENT_COUNT_SAMPLE = 5
+  TIME_PERIOD_DAYS = 90
+
+  attr_reader :project
+
+  def initialize(project)
+    @project = project
+  end
+
+  def seed!
+    create_ai_usage_data
+    sync_to_click_house
+  end
+
+  def create_ai_usage_data
+    project.users.count.times do
+      user = project.users.sample
+
+      CODE_PUSH_SAMPLE.times do
+        Event.create!(
+          project: project,
+          author: user,
+          action: :pushed,
+          created_at: rand(TIME_PERIOD_DAYS).days.ago
+        )
+      end
+
+      AI_EVENT_COUNT_SAMPLE.times do
+        event_data = {
+          user_id: user.id,
+          event: Gitlab::Tracking::AiTracking::EVENTS['code_suggestions_requested'],
+          timestamp: rand(TIME_PERIOD_DAYS).days.ago
+        }
+        ClickHouse::WriteBuffer.write_event(event_data)
+      end
+    end
+  end
+
+  def sync_to_click_house
+    ClickHouse::CodeSuggestionEventsCronWorker.new.perform
+
+    # Re-sync data with ClickHouse
+    ClickHouse::SyncCursor.update_cursor_for('events', 0)
+    Gitlab::ExclusiveLease.skipping_transaction_check do
+      ClickHouse::EventsSyncWorker.new.perform
+    end
+  end
+end
+
+Gitlab::Seeder.quiet do
+  feature_available = ::Gitlab::ClickHouse.globally_enabled_for_analytics? &&
+    Feature.enabled?(:clickhouse_data_collection) &&
+    Feature.enabled?(:event_sync_worker_for_click_house) &&
+    Feature.enabled?(:code_suggestion_events_in_click_house)
+
+  unless feature_available
+    puts "
+    WARNING:
+    To use this seed file, you need to make sure that ClickHouse is configured and enabled with your GDK.
+    Once you've configured the config/click_house.yml file, run the migrations:
+
+    > bundle exec rake gitlab:clickhouse:migrate
+
+    In a Rails console session, enable ClickHouse for analytics and the feature flags:
+
+    Gitlab::CurrentSettings.current_application_settings.update(use_clickhouse_for_analytics: true)
+
+    Feature.enable(:clickhouse_data_collection)
+    Feature.enable(:event_sync_worker_for_click_house)
+    Feature.enable(:code_suggestion_events_in_click_house)
+    "
+    break
+  end
+
+  projects = Project.all
+  projects = projects.where(id: ENV['PROJECT_ID']) if ENV['PROJECT_ID']
+
+  projects.find_each do |project|
+    seeder = Gitlab::Seeder::AiUsageStats.new(project)
+    seeder.seed!
+  end
+end
+# rubocop:enable Rails/Output