Geo: Add verification related database columns for MR diffs verification [RUN ALL RSPEC] [RUN AS-IF-FOSS] (!60935) · Merge requests · GitLab.org / GitLab

Aakriti Gupta requested to merge ag-verify-mr-diffs-new into master May 04, 2021

What does this MR do?

This MR adds verification columnsfor MR diffs:

These are based on the checklist in the new issue template for adding replication/verification to blob types.

!63309 (merged) uses these columns. The diffs are separated for easier reviewing.

Related issue: #323285 (closed)

Database review information

Migration outputs

Geo db migration

== 20210504143244 AddVerificationToMergeRequestDiffRegistry: migrating ========
-- add_column(:merge_request_diff_registry, :verification_started_at, :datetime_with_timezone)
   -> 0.0081s
-- add_column(:merge_request_diff_registry, :verified_at, :datetime_with_timezone)
   -> 0.0007s
-- add_column(:merge_request_diff_registry, :verification_retry_at, :datetime_with_timezone)
   -> 0.0006s
-- add_column(:merge_request_diff_registry, :verification_retry_count, :integer)
   -> 0.0009s
-- add_column(:merge_request_diff_registry, :verification_state, :integer, {:limit=>2, :default=>0, :null=>false})
   -> 0.0050s
-- add_column(:merge_request_diff_registry, :checksum_mismatch, :boolean)
   -> 0.0007s
-- add_column(:merge_request_diff_registry, :verification_checksum, :binary)
   -> 0.0007s
-- add_column(:merge_request_diff_registry, :verification_checksum_mismatched, :binary)
   -> 0.0005s
-- add_column(:merge_request_diff_registry, :verification_failure, :string, {:limit=>255})
   -> 0.0006s
== 20210504143244 AddVerificationToMergeRequestDiffRegistry: migrated (0.0182s)

➜  gitlab git:(ag-verify-mr-diffs-new) ✗ rake geo:db:migrate
== 20210505170208 AddIndexesToMergeRequestDiffRegistryTable: migrating ========
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:merge_request_diff_registry, :merge_request_diff_id, {:name=>"index_merge_request_diff_registry_on_t_state_version_id", :unique=>true, :algorithm=>:concurrently})
   -> 0.0030s
-- index_exists?(:merge_request_diff_registry, :verification_retry_at, {:name=>"merge_request_diff_registry_failed_verification", :order=>"NULLS FIRST", :where=>"((state = 2) AND (verification_state = 3))", :algorithm=>:concurrently})
   -> 0.0014s
-- add_index(:merge_request_diff_registry, :verification_retry_at, {:name=>"merge_request_diff_registry_failed_verification", :order=>"NULLS FIRST", :where=>"((state = 2) AND (verification_state = 3))", :algorithm=>:concurrently})
   -> 0.0034s
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:merge_request_diff_registry, :verification_state, {:name=>"merge_request_diff_registry_needs_verification", :where=>"((state = 2)  AND (verification_state = ANY (ARRAY[0, 3])))", :algorithm=>:concurrently})
   -> 0.0020s
-- add_index(:merge_request_diff_registry, :verification_state, {:name=>"merge_request_diff_registry_needs_verification", :where=>"((state = 2)  AND (verification_state = ANY (ARRAY[0, 3])))", :algorithm=>:concurrently})
   -> 0.0022s
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:merge_request_diff_registry, :verified_at, {:name=>"merge_request_diff_registry_pending_verification", :order=>"NULLS FIRST", :where=>"((state = 2) AND (verification_state = 0))", :algorithm=>:concurrently})
   -> 0.0019s
-- add_index(:merge_request_diff_registry, :verified_at, {:name=>"merge_request_diff_registry_pending_verification", :order=>"NULLS FIRST", :where=>"((state = 2) AND (verification_state = 0))", :algorithm=>:concurrently})
   -> 0.0026s
== 20210505170208 AddIndexesToMergeRequestDiffRegistryTable: migrated (0.0295s)

Db migration

== 20210504143128 AddVerificationStateAndStartedAtToMrDiffDetailsTable: migrating
-- change_table(:merge_request_diff_details)
   -> 0.0094s
== 20210504143128 AddVerificationStateAndStartedAtToMrDiffDetailsTable: migrated (0.0094s)

gitlab git:(ag-verify-mr-diffs-new) ✗ rake db:migrate
== 20210505170152 AddVerificationIndexesToMergeRequestDiffDetailsTable: migrating
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:merge_request_diff_details, :verification_state, {:name=>"index_merge_request_diff_details_on_verification_state", :algorithm=>:concurrently})
   -> 0.0049s
-- execute("SET statement_timeout TO 0")
   -> 0.0005s
-- add_index(:merge_request_diff_details, :verification_state, {:name=>"index_merge_request_diff_details_on_verification_state", :algorithm=>:concurrently})
   -> 0.0103s
-- execute("RESET ALL")
   -> 0.0005s
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:merge_request_diff_details, :verified_at, {:where=>"(verification_state = 0)", :order=>{:verified_at=>"ASC NULLS FIRST"}, :name=>"index_merge_request_diff_details_pending_verification", :algorithm=>:concurrently})
   -> 0.0014s
-- add_index(:merge_request_diff_details, :verified_at, {:where=>"(verification_state = 0)", :order=>{:verified_at=>"ASC NULLS FIRST"}, :name=>"index_merge_request_diff_details_pending_verification", :algorithm=>:concurrently})
   -> 0.0030s
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:merge_request_diff_details, :verification_retry_at, {:where=>"(verification_state = 3)", :order=>{:verification_retry_at=>"ASC NULLS FIRST"}, :name=>"index_merge_request_diff_details_failed_verification", :algorithm=>:concurrently})
   -> 0.0013s
-- add_index(:merge_request_diff_details, :verification_retry_at, {:where=>"(verification_state = 3)", :order=>{:verification_retry_at=>"ASC NULLS FIRST"}, :name=>"index_merge_request_diff_details_failed_verification", :algorithm=>:concurrently})
   -> 0.0022s
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:merge_request_diff_details, :verification_state, {:where=>"(verification_state = 0 OR verification_state = 3)", :name=>"index_merge_request_diff_details_needs_verification", :algorithm=>:concurrently})
   -> 0.0013s
-- add_index(:merge_request_diff_details, :verification_state, {:where=>"(verification_state = 0 OR verification_state = 3)", :name=>"index_merge_request_diff_details_needs_verification", :algorithm=>:concurrently})
   -> 0.0021s
== 20210505170152 AddVerificationIndexesToMergeRequestDiffDetailsTable: migrated (0.0307s)

Rollbacks

➜  gitlab git:(ag-verify-mr-diffs-new) rake db:rollback STEP=2
== 20210505170152 AddVerificationIndexesToMergeRequestDiffDetailsTable: reverting
-- transaction_open?()
   -> 0.0000s
-- indexes(:merge_request_diff_details)
   -> 0.0029s
-- execute("SET statement_timeout TO 0")
   -> 0.0004s
-- remove_index(:merge_request_diff_details, {:algorithm=>:concurrently, :name=>"index_merge_request_diff_details_on_verification_state"})
   -> 0.0026s
-- execute("RESET ALL")
   -> 0.0005s
-- transaction_open?()
   -> 0.0000s
-- indexes(:merge_request_diff_details)
   -> 0.0014s
-- remove_index(:merge_request_diff_details, {:algorithm=>:concurrently, :name=>"index_merge_request_diff_details_pending_verification"})
   -> 0.0018s
-- transaction_open?()
   -> 0.0000s
-- indexes(:merge_request_diff_details)
   -> 0.0012s
-- remove_index(:merge_request_diff_details, {:algorithm=>:concurrently, :name=>"index_merge_request_diff_details_failed_verification"})
   -> 0.0016s
-- transaction_open?()
   -> 0.0000s
-- indexes(:merge_request_diff_details)
   -> 0.0010s
-- remove_index(:merge_request_diff_details, {:algorithm=>:concurrently, :name=>"index_merge_request_diff_details_needs_verification"})
   -> 0.0017s
== 20210505170152 AddVerificationIndexesToMergeRequestDiffDetailsTable: reverted (0.0179s)

== 20210504143128 AddVerificationStateAndStartedAtToMrDiffDetailsTable: reverting
-- remove_column(:merge_request_diff_details, :verification_started_at, :datetime_with_timezone, {})
   -> 0.0017s
-- remove_column(:merge_request_diff_details, :verification_state, :integer, {:default=>0, :limit=>2, :null=>false})
   -> 0.0018s
== 20210504143128 AddVerificationStateAndStartedAtToMrDiffDetailsTable: reverted (0.0052s)

➜  gitlab git:(ag-verify-mr-diffs-new) ✗ rake geo:db:rollback STEP=2
== 20210505170208 AddIndexesToMergeRequestDiffRegistryTable: reverting ========
-- transaction_open?()
   -> 0.0000s
-- indexes(:merge_request_diff_registry)
   -> 0.0040s
-- remove_index(:merge_request_diff_registry, {:algorithm=>:concurrently, :name=>"index_merge_request_diff_registry_on_t_state_version_id"})
   -> 0.0024s
-- transaction_open?()
   -> 0.0000s
-- indexes(:merge_request_diff_registry)
   -> 0.0024s
-- remove_index(:merge_request_diff_registry, {:algorithm=>:concurrently, :name=>:index_merge_request_diff_registry_on_retry_at})
   -> 0.0013s
-- transaction_open?()
   -> 0.0000s
-- indexes(:merge_request_diff_registry)
   -> 0.0015s
-- remove_index(:merge_request_diff_registry, {:algorithm=>:concurrently, :name=>:index_merge_request_diff_registry_on_state})
   -> 0.0018s
-- transaction_open?()
   -> 0.0000s
-- indexes(:merge_request_diff_registry)
   -> 0.0013s
-- remove_index(:merge_request_diff_registry, {:algorithm=>:concurrently, :name=>"merge_request_diff_registry_failed_verification"})
   -> 0.0018s
-- transaction_open?()
   -> 0.0000s
-- indexes(:merge_request_diff_registry)
   -> 0.0022s
-- remove_index(:merge_request_diff_registry, {:algorithm=>:concurrently, :name=>"merge_request_diff_registry_needs_verification"})
   -> 0.0011s
-- transaction_open?()
   -> 0.0000s
-- indexes(:merge_request_diff_registry)
   -> 0.0012s
-- remove_index(:merge_request_diff_registry, {:algorithm=>:concurrently, :name=>"merge_request_diff_registry_pending_verification"})
   -> 0.0009s
== 20210505170208 AddIndexesToMergeRequestDiffRegistryTable: reverted (0.0265s)

== 20210504143244 AddVerificationToMergeRequestDiffRegistry: reverting ========
-- remove_column(:merge_request_diff_registry, :verification_failure, :string, {:limit=>255})
   -> 0.0017s
-- remove_column(:merge_request_diff_registry, :verification_checksum_mismatched, :binary)
   -> 0.0006s
-- remove_column(:merge_request_diff_registry, :verification_checksum, :binary)
   -> 0.0006s
-- remove_column(:merge_request_diff_registry, :checksum_mismatch, :boolean)
   -> 0.0006s
-- remove_column(:merge_request_diff_registry, :verification_state, :integer, {:limit=>2, :default=>0, :null=>false})
   -> 0.0007s
-- remove_column(:merge_request_diff_registry, :verification_retry_count, :integer)
   -> 0.0013s
-- remove_column(:merge_request_diff_registry, :verification_retry_at, :datetime_with_timezone)
   -> 0.0006s
-- remove_column(:merge_request_diff_registry, :verified_at, :datetime_with_timezone)
   -> 0.0006s
-- remove_column(:merge_request_diff_registry, :verification_started_at, :datetime_with_timezone)
   -> 0.0012s
== 20210504143244 AddVerificationToMergeRequestDiffRegistry: reverted (0.0100s)

Screenshots (strongly suggested)

Does this MR meet the acceptance criteria?

Conformity

📋 Does this MR need a changelog?
- I have included a changelog entry.
- I have not included a changelog entry because this change is not user facing.
[-] Documentation (if required)
Code review guidelines
Merge request performance guidelines
Style guides
Database guides
Separation of EE specific content

Edited Jun 04, 2021 by Aakriti Gupta

Geo: Add verification related database columns for MR diffs verification [RUN ALL RSPEC] [RUN AS-IF-FOSS]