Skip to content

Track weight and sprint_id for VSA issues

Adam Hegyi requested to merge 419925-track-weight-and-sprint-for-vsa-issues into master

What does this MR do and why?

This MR adds the weight and sprint_id columns to VSA so we can add them as a filter. Note: VSA works on aggregated data tables where various columns are pulled from the source table (issues, merge_requests) and stored in a way that we can easily filter them.

Not: this is not a user-facing change yet. Adding the filters will happen in a follow-up.

Database

The analytics_cycle_analytics_issue_stage_events table does not have foreign keys (so we ignore the check for sprint id in the tests), consistency is enforced with periodical background jobs (more context: https://docs.gitlab.com/ee/development/value_stream_analytics.html#database-query-backend)

Up:

main: == [advisory_lock_connection] object_id: 231860, pg_backend_pid: 1026518
main: == 20230918123357 AddSprintIdAndWeightToVsaIssues: migrating ==================
main: -- add_column(:analytics_cycle_analytics_issue_stage_events, :weight, :integer, {:limit=>4})
main:    -> 0.0036s
main: -- add_column(:analytics_cycle_analytics_issue_stage_events, :sprint_id, :integer, {:limit=>8})
main:    -> 0.0022s
main: == 20230918123357 AddSprintIdAndWeightToVsaIssues: migrated (0.0101s) =========
main: == [advisory_lock_connection] object_id: 231860, pg_backend_pid: 1026518

ci: == [advisory_lock_connection] object_id: 232160, pg_backend_pid: 1026521
ci: == 20230918123357 AddSprintIdAndWeightToVsaIssues: migrating ==================
ci: -- add_column(:analytics_cycle_analytics_issue_stage_events, :weight, :integer, {:limit=>4})
ci:    -> 0.0255s
ci: -- add_column(:analytics_cycle_analytics_issue_stage_events, :sprint_id, :integer, {:limit=>8})
ci:    -> 0.0017s
ci: == 20230918123357 AddSprintIdAndWeightToVsaIssues: migrated (0.0381s) =========

Down:

main: == [advisory_lock_connection] object_id: 226880, pg_backend_pid: 1023889
main: == 20230918123357 AddSprintIdAndWeightToVsaIssues: reverting ==================
main: -- remove_column(:analytics_cycle_analytics_issue_stage_events, :sprint_id)
main:    -> 0.0041s
main: -- remove_column(:analytics_cycle_analytics_issue_stage_events, :weight)
main:    -> 0.0014s
main: == 20230918123357 AddSprintIdAndWeightToVsaIssues: reverted (0.0101s) =========
main: == [advisory_lock_connection] object_id: 226880, pg_backend_pid: 1023889

ci: == [advisory_lock_connection] object_id: 226800, pg_backend_pid: 1024844
ci: == 20230918123357 AddSprintIdAndWeightToVsaIssues: reverting ==================
ci: -- remove_column(:analytics_cycle_analytics_issue_stage_events, :sprint_id)
ci:    -> 0.0065s
ci: -- remove_column(:analytics_cycle_analytics_issue_stage_events, :weight)
ci:    -> 0.0016s
ci: == 20230918123357 AddSprintIdAndWeightToVsaIssues: reverted (0.0186s) =========

How to set up and validate locally

  1. Ensure that you're on ultimate license plan.
  2. Ensure that VSA is seeded: FILTER=cycle_analytics SEED_VSA=1 bundle exec rake db:seed_fu
  3. Pick a few issues in the generated project and update the weight
      Project.find_by_path("vsmp-1695043691").issues.sample(3).each {|issue| issue.update!(weight: rand(5)) }
  4. Invoke the periodical aggregation worker:
    Analytics::CycleAnalytics::Aggregation.update_all(enabled: true)
    Analytics::CycleAnalytics::IncrementalWorker.new.perform
    puts Analytics::CycleAnalytics::IssueStageEvent.pluck(:weight).compact # we should see some values collected

-->

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #419925 (closed)

Edited by Adam Hegyi

Merge request reports