Track unique visitors to analytics pages
What does this MR do?
Solves https://gitlab.com/gitlab-org/telemetry/-/issues/380, together with adding data to usage ping (original issue: #221075 (closed) and approved MR: !34732 (merged))
This MR tracks unique users visiting listed analytics pages (only if the format is HTML, to avoid tracking API calls and only when the usage ping is enabled) by storing a long-lived cookie (24 months) containing a random UUID. In that way, we can track visits without associating them to the user id. We do not track visits if DNT is enabled by the user.
We also get the count of unique visits on any target.
The counts are added to the usage ping. Once this MR https://gitlab.com/gitlab-services/version-gitlab-com/-/merge_requests/485 is merged, we will be able to see unique visits data in the usage ping.
This MR covers listed pages with assigned target_id name:
| page | target id |
|---|---|
| /groups/:group/-/contribution_analytics | g_analytics_contribution |
| /groups/:group/-/insights | g_analytics_insights |
| /groups/:group/-/issues_analytics | g_analytics_issues |
| /groups/:group/-/analytics/productivity_analytics | g_analytics_productivity |
| /groups/:group/-/analytics/value_stream_analytics | g_analytics_valuestream |
| /:group/:project/pipelines/charts | p_analytics_pipelines |
| /:group/:project/-/analytics/code_reviews | p_analytics_code_reviews |
| /:group/:project/-/value_stream_analytics | p_analytics_valuestream |
| /:group/:project/insights | p_analytics_insights |
| /:group/:project/-/analytics/issues_analytics | p_analytics_issues |
| /:group/:project/-/graphs/master/charts | p_analytics_repo |
| /dashboard/todos | u_analytics_todos |
| /-/instance_statistics/cohorts | i_analytics_cohorts |
| /-/instance_statistics/dev_ops_score | i_analytics_dev_ops_score |
Note on implementation choice: I created a service with methods for tracking visits and counting them to separate the interface from the storage. The count of unique visits is implemented with Redis HyperLogLog, for efficiency, stale keys are automatically expired after 28 days. We track weekly unique visits where the week bucket is as defined by ISO 8601:
ISO 8601 week-based year and week number: The week 1 of YYYY starts with a Monday and includes YYYY-01-04. The days in the year before the first week are in the last week of the previous year. %G - The week-based year %g - The last 2 digits of the week-based year (00..99) %V - Week number of the week-based year (01..53)
This differs from the US convention of starting the week on Sunday but has the important advantage of having each week of exactly 7 days. (Other methods result in partial weeks at the beginning or the end of the year.)
Screenshots
Does this MR meet the acceptance criteria?
Conformity
-
Changelog entry -
Documentation (if required) -
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides -
Separation of EE specific content
Availability and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. -
Tested in all supported browsers -
Informed Infrastructure department of a default or new setting change, if applicable per definition of done
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
-
Label as security and @ mention @gitlab-com/gl-security/appsec -
The MR includes necessary changes to maintain consistency between UI, API, email, or other methods -
Security reports checked/validated by a reviewer from the AppSec team