Skip to content

Add monthly data to redis_hll_counters in usage data

What does this MR do?

We add automatically in usage data payload all known events with time frame last week #241686 (closed)

We could also add the monthly time frame to have the data for last 28 days(for daily aggregation) and last 4 complete weeks weekly aggregation

cc @gitlab-org/growth/telemetry/engineers

json for this currently:

{:redis_hll_counters=>
  {"compliance"=>{"g_compliance_dashboard"=>0, "g_compliance_audit_events"=>0, "i_compliance_audit_events"=>0, "i_compliance_credential_inventory"=>0, "compliance_total_unique_counts_weekly"=>0, "compliance_total_unique_counts_monthly"=>0},
   "analytics"=>
    {"g_analytics_contribution"=>0,
     "g_analytics_insights"=>0,
     "g_analytics_issues"=>0,
     "g_analytics_productivity"=>0,
     "g_analytics_valuestream"=>0,
     "p_analytics_pipelines"=>0,
     "p_analytics_code_reviews"=>0,
     "p_analytics_valuestream"=>0,
     "p_analytics_insights"=>0,
     "p_analytics_issues"=>0,
     "p_analytics_repo"=>0,
     "i_analytics_cohorts"=>0,
     "i_analytics_dev_ops_score"=>0,
     "g_analytics_merge_request"=>0,
     "p_analytics_merge_request"=>0,
     "analytics_total_unique_counts_weekly"=>0,
     "analytics_total_unique_counts_monthly"=>0},
   "ide_edit"=>{"g_edit_by_web_ide"=>0, "g_edit_by_sfe"=>0, "g_edit_by_snippet_ide"=>0, "ide_edit_total_unique_counts_weekly"=>0, "ide_edit_total_unique_counts_monthly"=>0},
   "search"=>{"i_search_total"=>0, "i_search_advanced"=>0, "i_search_paid"=>0, "search_total_unique_counts_weekly"=>0, "search_total_unique_counts_monthly"=>0},
   "source_code"=>{"wiki_action"=>0, "design_action"=>0, "project_action"=>0, "merge_request_action"=>0, "code_intelligence"=>0}}}

With this MR

  • Add counts_monthly counts_weekly keys
  • Add categories key
  • Add total_unique_counts for each categegory
  • Add monthly data for all know_events

redis_hll_counters will become

=> {:redis_hll_counters=>
  {"compliance"=>
    {"g_compliance_dashboard_weekly"=>0,
     "g_compliance_dashboard_monthly"=>0,
     "g_compliance_audit_events_weekly"=>0,
     "g_compliance_audit_events_monthly"=>0,
     "i_compliance_audit_events_weekly"=>0,
     "i_compliance_audit_events_monthly"=>0,
     "i_compliance_credential_inventory_weekly"=>0,
     "i_compliance_credential_inventory_monthly"=>0,
     "a_compliance_audit_events_api_weekly"=>0,
     "a_compliance_audit_events_api_monthly"=>0,
     "compliance_total_unique_counts_weekly"=>0,
     "compliance_total_unique_counts_monthly"=>0},
   "analytics"=>
    {"g_analytics_contribution_weekly"=>0,
     "g_analytics_contribution_monthly"=>0,
     "g_analytics_insights_weekly"=>0,
     "g_analytics_insights_monthly"=>0,
     "g_analytics_issues_weekly"=>0,
     "g_analytics_issues_monthly"=>0,
     "g_analytics_productivity_weekly"=>0,
     "g_analytics_productivity_monthly"=>0,
     "g_analytics_valuestream_weekly"=>0,
     "g_analytics_valuestream_monthly"=>0,
     "p_analytics_pipelines_weekly"=>0,
     "p_analytics_pipelines_monthly"=>0,
     "p_analytics_code_reviews_weekly"=>0,
     "p_analytics_code_reviews_monthly"=>0,
     "p_analytics_valuestream_weekly"=>0,
     "p_analytics_valuestream_monthly"=>0,
     "p_analytics_insights_weekly"=>0,
     "p_analytics_insights_monthly"=>0,
     "p_analytics_issues_weekly"=>0,
     "p_analytics_issues_monthly"=>0,
     "p_analytics_repo_weekly"=>0,
     "p_analytics_repo_monthly"=>0,
     "i_analytics_cohorts_weekly"=>0,
     "i_analytics_cohorts_monthly"=>0,
     "i_analytics_dev_ops_score_weekly"=>0,
     "i_analytics_dev_ops_score_monthly"=>0,
     "g_analytics_merge_request_weekly"=>0,
     "g_analytics_merge_request_monthly"=>0,
     "p_analytics_merge_request_weekly"=>0,
     "p_analytics_merge_request_monthly"=>0,
     "i_analytics_instance_statistics_weekly"=>0,
     "i_analytics_instance_statistics_monthly"=>0,
     "analytics_total_unique_counts_weekly"=>0,
     "analytics_total_unique_counts_monthly"=>0},
   "ide_edit"=>
    {"g_edit_by_web_ide_weekly"=>0,
     "g_edit_by_web_ide_monthly"=>0,
     "g_edit_by_sfe_weekly"=>0,
     "g_edit_by_sfe_monthly"=>0,
     "g_edit_by_snippet_ide_weekly"=>0,
     "g_edit_by_snippet_ide_monthly"=>0,
     "ide_edit_total_unique_counts_weekly"=>0,
     "ide_edit_total_unique_counts_monthly"=>0},
   "search"=>
    {"i_search_total_weekly"=>0,
     "i_search_total_monthly"=>0,
     "i_search_advanced_weekly"=>0,
     "i_search_advanced_monthly"=>0,
     "i_search_paid_weekly"=>0,
     "i_search_paid_monthly"=>0,
     "search_total_unique_counts_weekly"=>0,
     "search_total_unique_counts_monthly"=>0},
   "source_code"=>
    {"wiki_action_weekly"=>0,
     "wiki_action_monthly"=>0,
     "design_action_weekly"=>0,
     "design_action_monthly"=>0,
     "project_action_weekly"=>0,
     "project_action_monthly"=>0,
     "merge_request_action_weekly"=>0,
     "merge_request_action_monthly"=>0,
     "i_source_code_code_intelligence_weekly"=>0,
     "i_source_code_code_intelligence_monthly"=>0},
....

Screenshots

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team

Closes #247098 (closed)

Edited by Alina Mihaila

Merge request reports