Add Snowplow buffering

What does this MR do and why?

Related to #508460 (closed), #517359 (closed)

We want to test out buffer sizes with Snowplow.

References

Please include cross links to any resources that are relevant to this MR. This will give reviewers and future readers helpful context to give an efficient review of the changes introduced.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

How to set up and validate locally

  1. Run Snowplow Micro locally: https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/doc/howto/snowplow_micro.md
  2. Enable the feature flags: Feature.enable(:snowplow_tracking_post_method); Feature.enable(:snowplow_buffer_events)
  3. Restart the rails console. This is to make sure the Snowplow class instance gets initiated again.
  4. Trigger an event a few (less than 100) times, for example by running this in the rails console: 3.times { Gitlab::InternalEvents.track_event('use_run_keyword_in_cicd_yaml', user: User.first) }
  5. Make sure that the event have NOT been recorded by Snowplow Micro: http://127.0.0.1:9091/micro/ui
  6. Trigger the events again, this time making sure you've passed 10 events: 10.times { Gitlab::InternalEvents.track_event('use_run_keyword_in_cicd_yaml', user: User.first) }
  7. Make sure that the a batch of 100 events has been recorded by Snowplow Micro: http://127.0.0.1:9091/micro/ui
  8. Now, test the at_exit hook: make sure that you have triggered some events that have not been sent yet [eg. when following instructions supplied in the previous steps, you should have 3 batched unsent events]. Then, close the console, eg. by running exit or ctrl+d.
  9. Make sure that the leftover events have been sent to Snowplow: http://127.0.0.1:9091/micro/ui
Edited by Jonas Larsen

Merge request reports

Loading