Cycle Analytics: first iteration

What does this MR do?

• Implement the first iteration of the "Cycle Analytics" feature.

What are the relevant issue numbers?

• Closes #21170 (closed)

Does this MR have an associated MR on Gitlab EE?

• gitlab-org/gitlab-ee!748

Screenshots

Backend Tasks

• Implementation
  • Phases
    • Issue (Tracker)
    • Plan (Board)
    • Code (IDE)
    • Test (CI)
    • Review (MR)
    • Staging (CD)
    • Production (Total)
  • Make heuristics more modular
  • Scope to project
  • Date range (30 days, 90 days)
  • Access restriction
• Test
  • Find a better way to test these phases
  • Phases
    • Issue (Tracker)
    • Plan (Board)
    • Code (IDE)
    • Test (CI)
    • Review (MR)
    • Staging (CD)
    • Production (Total)
  • Test for "end case happens before start case"
  • Consolidate helper
• Miniboss review
• Performance testing with mock data
• Improve performance
  • Pre-calculate "merge requests closing issues
  • Pre-calculate everything else
• Test performance against 10k issues
• Test all pre-calculation code
  • Ci::Pipeline -> build start/finish
  • Ci::Pipeline#merge_requests
  • Issue -> record default metrics after save
  • MergeRequest -> record default metrics after save
  • Deployment -> Update "first_deployed_to_production_at" for MR metrics
  • Git Push -> Update "first commit mention" for issue metrics
  • Merge request create/update/refresh -> Update "merge requests closing issues"
• Remove MergeRequestsClosingIssues when necessary
• Changes to unblock Fatih
  • Add summary data
  • stats should be array
  • Let stats be null if all stats are null
• Indexes for "merge requests closing issues"
• Test summary data
• Scope everything to project
  • Find out why tests were passing
• Filter should include issues/MRs which have made it to production within the range
• Don't create duplicate MergeRequestsClosingIssues
• Fix tests
• MySQL median
• Assign to Douwe for review
• Fix conflicts
• Implement suggestions from Yorick's review
  • Test on PG
  • Test on MySQL
• Refactor
  • Cleanup
    • What happens if we have no data at all?
    • Extract common queries to methods / scopes
  • Remove unused queries
  • Downtime for foreign key migrations
  • Find a way around "if issue.metrics.present?" all over the place
  • Find a way around "if merge_request.metrics.present?" all over the place
  • Test migrations on a fresh database
    • MySQL
    • Pg
• Access issues
  • While the project is public and the visibility is set to "Everyone with access", you cannot visit the cycle analytics page when signed out.
• CHANGELOG
• Implement suggestions from Douwe's review
  • First set of comments
  • Second set of comments
  • Third set of comments
  • Fourth set of comments
• Make sure build is green
• Wait for merge
• EE MR

mentioned in issue #21170 (closed)

Added 1 commit:

• 155b5ff4 - Add the "Plan" Cycle Analytics query.

Added 2 commits:

• 27fcf72c - Add the "Code" Cycle Analytics section.
• a232ab5b - Add the "Review" cycle analytics section.

Added 2 commits:

• 6ce65bc6 - Add the "Staging" cycle analytics section.
• bcf5bf25 - Add the "Production" cycle analytics section.

Added 1 commit:

• e75277cd - Add the "Test" cycle analytics section.

Added 1 commit:

• 34d3fefc - Allow multiple queries for each cycle analytics section.

Marked the task Make heuristics more modular as completed

Added 1 commit:

• 54e0befc - Add a spec for the CycleAnalytics#issue method.

Added 2 commits:

• 22ec9082 - Scope Cycle Analytics queries to a project
• b8cb23a8 - Fix minor error in 'CycleAnalytics#show'

Added 1 commit:

• 44efc056 - Fetch cycle analytics data for a specific date range.

Added 1 commit:

• 2a510e19 - Authorize access to cycle analytics.

Marked the task Issue (Tracker) as completed

Marked the task Test query as completed

Marked the task Access restriction as completed

Added 168 commits:

• 2a510e19...1bf2fe27 - 154 commits from branch master
• 516c838a - Add an Issue::Metrics model.
• 8ccea81c - Display numbers for the "Issue (Tracker)" cycle analytics metric.
• f932bb8e - Add the "Plan" Cycle Analytics query.
• 487906b3 - Add the "Code" Cycle Analytics section.
• 14d6317e - Add the "Review" cycle analytics section.
• d3fef0fb - Add the "Staging" cycle analytics section.
• 3d5729a7 - Add the "Production" cycle analytics section.
• fc92d066 - Add the "Test" cycle analytics section.
• ebc03833 - Allow multiple queries for each cycle analytics section.
• a81de9ab - Add a spec for the CycleAnalytics#issue method.
• df6c9c33 - Scope Cycle Analytics queries to a project
• ce6bcdd0 - Fix minor error in 'CycleAnalytics#show'
• 331080bc - Fetch cycle analytics data for a specific date range.
• 2f57eee7 - Authorize access to cycle analytics.

Added 3 commits:

• 0910868d - Tweak cycle analytics query to match the current requirements.
• 0f748605 - Add tests for the plan cycle analytics phase.
• 98c9d120 - Refactor cycle analytics specs.

Marked the task Plan (Board) as completed

Marked the task Test query as completed

Added 1 commit:

• 538958be - Test the code cycle analytics phase.

Added 3 commits:

• f77c952a - Test the code cycle analytics phase.
• bdd3a5b9 - Test the 'review' cycle analytics phase.
• 992b7720 - Generate branch/file names more randomly to avoid failures.