Document differences between Gitlab::Metrics and Gitlab::Tracking with some use cases
Problem to solve
While I was looking into #31286 (closed), I came accross two methods (Gitlab::Metrics and Gitlab::Tracking) we can use to track custom events. I've asked around to get a clarity around when to use what, but didn't get a clear response so far.
It'd be good if we can consolidate our understanding on this matter and document it for the sake of clarity.
Further details
Here are the two places where it talks a bit about each methods.
- https://docs.gitlab.com/ee/development/event_tracking/backend.html.
- https://docs.gitlab.com/ee/development/instrumentation.html (mostly talking about performance tracking, but it also mentions custom event tracking at the bottom.
The difference seems to be that one (Gitlab::Metrics.add_event) goes to InfluxDB, another goes to snowplow. However, it's unclear what are the pros and cons of each methods and the example use cases.
Here are some explanations given by @a_akgun vis Slack.
Here are some types of events, trackings I've faced: there maybe more:
- Snowplow frontend and backend event tracking (for user events and backend events) https://docs.gitlab.com/ee/development/event_tracking/
- Influxdb events https://docs.gitlab.com/ee/development/instrumentation.html (timeseries db, currently used for instrumentation and performance tracking)
- Logs and warning/error https://docs.gitlab.com/ee/administration/logs.html (warnings and errors)
- Audit events https://docs.gitlab.com/ee/administration/audit_events.html Compliance and security
- app\models\event.rb is also a type of event for user activities (whatever you see on your user timeline)
- and usage ping for self-hosted instanced aggregate counts
When to use what? My own vague understanding which may evolve but help
- Snowplow tracking: it goes to the data teams's Snowflake and is durable/persistent which can give you nice Periscope charts for gitlab.com (but not used for self-hosted unless the admins install snow*)
- app\models\even.rb` whenever activity is visible for a user on their profile for example
- usage ping for self-hosted counts
- audit events for security related instance admin compliance events.
- logs/errors/warnings as usual
- influxdb : performance related timeseries data, retention duration and policy for gitlab.com or others maybe low for stats
Proposal
I suppose we can add more detailed information about the differences and use cases. https://docs.gitlab.com/ee/development/event_tracking
Also, list https://docs.gitlab.com/ee/development/instrumentation.html in the doc as a side bar item if it is something that we want to continue to use as it is currently not listed.
Who can address the issue
Team members who have worked on either Gitlab::Metrics or Gitlab::Tracking.