Skip to content
Snippets Groups Projects

Fix storing first_mentioned_in_commit_at attribute

Merged Adam Hegyi requested to merge 339508-fix-recording-first_mentioned_in_commit_at into master
All threads resolved!

What does this MR do and why?

This change fixes the way we calculate and store the first_mentioned_in_commit_at attribute. Instead of looking at the most recent commit date, the system will check the earlies commit date looking back to a maximum of 100 commits. The change also includes a BG migration that attempts to fix the existing records.

BG migration:

  1. Batch over issue_metrics via issue_id where first_mentioned_in_commit_at was set later than 2019
  2. For each batch, do sub-batching. For each issue_metrics_record:
  3. Find merge_requests_closing_isues records related to the issue.
  4. Look up the MR diffs, look at the most recent diff.
  5. Look at the diff commits, take the authored_date column.
  6. If the found authored_date is earlier than first_mentioned_in_commit_at, then update the issue_metrics row.

Database

Queries

Migration

The index creation took 2 minutes on db-lab.

Up:

== 20211004110500 AddTemporaryIndexToIssueMetrics: migrating ==================
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:issue_metrics, :issue_id, {:where=>"EXTRACT(YEAR FROM first_mentioned_in_commit_at) > 2019", :name=>"index_issue_metrics_first_mentioned_in_commit", :algorithm=>:concurrently})
   -> 0.0030s
-- execute("SET statement_timeout TO 0")
   -> 0.0006s
-- add_index(:issue_metrics, :issue_id, {:where=>"EXTRACT(YEAR FROM first_mentioned_in_commit_at) > 2019", :name=>"index_issue_metrics_first_mentioned_in_commit", :algorithm=>:concurrently})
   -> 0.0070s
-- execute("RESET statement_timeout")
   -> 0.0006s
== 20211004110500 AddTemporaryIndexToIssueMetrics: migrated (0.0161s) =========

== 20211004110927 ScheduleFixFirstMentionedInCommitAtJob: migrating ===========
-- Scheduled 1 FixFirstMentionedInCommitAt jobs with a maximum of 10000 records per batch and an interval of 120 seconds.

The migration is expected to take at least 120 seconds. Expect all jobs to have completed after 2021-10-04 17:42:20 UTC."
== 20211004110927 ScheduleFixFirstMentionedInCommitAtJob: migrated (0.0147s) ==

Down:

== 20211004110927 ScheduleFixFirstMentionedInCommitAtJob: reverting ===========
== 20211004110927 ScheduleFixFirstMentionedInCommitAtJob: reverted (0.0000s) ==


== 20211004110500 AddTemporaryIndexToIssueMetrics: reverting ==================
-- transaction_open?()
   -> 0.0000s
-- indexes(:issue_metrics)
   -> 0.0025s
-- execute("SET statement_timeout TO 0")
   -> 0.0005s
-- remove_index(:issue_metrics, {:algorithm=>:concurrently, :name=>"index_issue_metrics_first_mentioned_in_commit"})
   -> 0.0059s
-- execute("RESET statement_timeout")
   -> 0.0006s
== 20211004110500 AddTemporaryIndexToIssueMetrics: reverted (0.0134s) =========

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #339508 (closed)

Edited by Adam Hegyi

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • removed review request for @m_frankiewicz

  • Alex Buijs approved this merge request

    approved this merge request

  • added databasereviewed label and removed databasereview pending label

  • Alex Buijs requested review from @tigerwnz and removed review request for @alexbuijs

    requested review from @tigerwnz and removed review request for @alexbuijs

  • :wave: @alexbuijs, thanks for approving this merge request.

    This is the first time the merge request is approved. To ensure full test coverage, a new pipeline has been started.

    For more info, please refer to the following links:

  • Tiger Watson approved this merge request

    approved this merge request

  • added databaseapproved label and removed databasereviewed label

  • Tiger Watson removed review request for @tigerwnz

    removed review request for @tigerwnz

  • Tiger Watson requested review from @ck3g

    requested review from @ck3g

  • Vitali Tatarintev approved this merge request

    approved this merge request

  • Vitali Tatarintev resolved all threads

    resolved all threads

  • Vitali Tatarintev enabled an automatic merge when the pipeline for 825630b5 succeeds

    enabled an automatic merge when the pipeline for 825630b5 succeeds

  • mentioned in commit ac1c0694

  • added workflowcanary label and removed workflowstaging label

  • added workflowproduction label and removed workflowcanary label

  • Adding index only. No impact to data warehouse

  • mentioned in issue #343724 (closed)

  • Ben Prescott_ mentioned in issue #345416

    mentioned in issue #345416

  • Adam Hegyi mentioned in merge request !81409 (merged)

    mentioned in merge request !81409 (merged)

  • Sean McGivern mentioned in merge request !113303 (merged)

    mentioned in merge request !113303 (merged)

  • Please register or sign in to reply
    Loading