Add service ping context to redis_hll_counters.ecosystem.ecosystem_total_unique_counts_monthly
Background
There are two Product Intelligence data tracking systems in place at GitLab: Service Ping and Snowplow event tracking. Recently there was effort undertaken by ~"group::product intelligence" to mirror some of data reported by Service Ping with Snowplow event tracking to help GitLab members better understand wider community interactions with GitLab, and make better decision how to develop and improve it.
Problem
The snowplow events labeled with:
-
redis_hll_counters.ecosystem.ecosystem_total_unique_counts_monthly
does not pass the required context yet to be usable for the data team.
Desired Outcome
Snowplow event custom context (see documentation about what context are) as defined here is passed to the event.
Implementation Guide
First locate all places where redis_hll_counters.ecosystem.ecosystem_total_unique_counts_monthly
is emitted (hint: see how Snowplow frontend and backend tracking is implemented at GitLab
Identify what Service Ping RedisHll events were mirrored for example at https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/models/integrations/base_slack_notification.rb#L45
def log_usage(event, user_id)
return unless user_id
return unless SUPPORTED_EVENTS_FOR_USAGE_LOG.include?(event)
key = "#{metrics_key_prefix}_#{event}_notification"
Gitlab::UsageDataCounters::HLLRedisCounter.track_event(key, values: user_id)
....
Via key
variable stores tracked event
Fill out required context with eg: https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/tracking/service_ping_context.rb for backend events
Gitlab::Tracking.event(
self.class.to_s,
name,
namespace: tracking_namespace_source,
user: current_user,
context: [Gitlab::Tracking::ServicePingContext.new(data_source: :redis_hll, event: name).to_context]
)
Or create context manually for front end events eg: https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/assets/javascripts/projects/pipelines/charts/components/app.vue#L85
this.track('click_tab', {
label: SNOWPLOW_LABEL,
context: {
schema: SNOWPLOW_SCHEMA,
data: {
event_name: event,
data_source: SNOWPLOW_DATA_SOURCE,
},
},
});
Test you changes with Snowplow Micro included in gdk
Make sure that all events from metric yaml definition file has been tracked with snowplow context
Example MR that achieves similar result for different event for reference: !103534 (diffs)
In case of questions head to #g_product_intelligence
slack channel or tag @gitlab-org/analytics-section/product-intelligence/engineers
in comment in this issue
Testing
Update related tests