Skip to content

Fix props in internal events sent via Product Analytics Ruby SDK

Piotr Skorupa requested to merge psk-fix-props-in-backend-internal-events into master

What does this MR do and why?

In the course of rolling out #432640 (closed), the changes behind the feature flag sent backend internal events to the Snowplow collector successfully, but failed to arrive in ClickHouse. Turns out they failed enrichment and they got persisted as Snowplow bad events in ClickHouse instead, with the following error:

{\"message\":\"$.props: null found, object expected\",\"path\":\"$.props\",\"keyword\":\"type\",\"targets\":[\"null\",\"object\"]}

This MR fixes the above error - Snowplow collector complaining about props being nil and violating the custom_event schema. We're now also passing extra info (project and namespace ids) for better context to the data.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

How to set up and validate locally

  1. Set up Product Analytics devkit to be able to receive the events into ClickHouse.
  2. Enable Product Analytics for a project in your GDK instance, for example gitlab-org/gitlab-shell by visiting <GDK-ROOT>/gitlab-org/gitlab-shell/-/analytics/dashboards/product-analytics-onboarding
  3. Set GITLAB_ANALYTICS_ID and GITLAB_ANALYTICS_URL to values provided after Product Analytics onboarding for that project.
  4. Send an event supported by the internal events framework (such as changing an issue description, closing/reopening or locking an issue).
  5. Visit ClickHouse query interface at http://localhost:8123/play and run select * from gitlab_project_<PROJECT_ID>.snowplow_events where useragent = 'GitLab Analytics Ruby SDK/0.2.3' to see the events.
  6. If you see no events, try disabling ad-block plugins and browser tracking protection.

Compare with the state before the changes - the event doesn't arrive at the gitlab_project(...).snowplow_events table, but instead there's a new row in default.snowplow_bad_events.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Piotr Skorupa

Merge request reports