Geo: Add verification state machine
What does this MR do?
- Add a state machine to manage verification state on the primary
- Make VerificationWorker use the state machine to update state
- Query records by state to produce success/failed counts
- This MR prepares us for running workers which constantly query for batches of records that need to be checksummed, and checksums many records simultaneously (see #13839 (closed) for more)
Part of #13839 (closed)
Database migrations
➜ gitlab git:(mk/add-verification-state-machine) ✗ bin/rake db:migrate
== 20201112173532 AddVerificationStateToPackageFiles: migrating ===============
-- add_column(:packages_package_files, :verification_state, :integer, {:default=>0, :limit=>2})
-> 0.0038s
-- add_column(:packages_package_files, :verification_started_at, :datetime_with_timezone)
-> 0.0016s
== 20201112173532 AddVerificationStateToPackageFiles: migrated (0.0055s) ======
== 20201112173911 AddIndexOnVerificationStateOnPackageFiles: migrating ========
-- transaction_open?()
-> 0.0000s
-- index_exists?(:packages_package_files, :verification_state, {:name=>"index_packages_package_files_on_verification_state", :algorithm=>:concurrently})
-> 0.0026s
-- execute("SET statement_timeout TO 0")
-> 0.0001s
-- add_index(:packages_package_files, :verification_state, {:name=>"index_packages_package_files_on_verification_state", :algorithm=>:concurrently})
-> 0.0035s
-- execute("RESET ALL")
-> 0.0002s
== 20201112173911 AddIndexOnVerificationStateOnPackageFiles: migrated (0.0068s)
➜ gitlab git:(mk/add-verification-state-machine) ✗ bin/rake db:rollback STEP=2
== 20201112173911 AddIndexOnVerificationStateOnPackageFiles: reverting ========
-- transaction_open?()
-> 0.0000s
-- indexes(:packages_package_files)
-> 0.0036s
-- execute("SET statement_timeout TO 0")
-> 0.0001s
-- remove_index(:packages_package_files, {:algorithm=>:concurrently, :name=>"index_packages_package_files_on_verification_state"})
-> 0.0021s
-- execute("RESET ALL")
-> 0.0002s
== 20201112173911 AddIndexOnVerificationStateOnPackageFiles: reverted (0.0064s)
== 20201112173532 AddVerificationStateToPackageFiles: reverting ===============
-- remove_column(:packages_package_files, :verification_started_at, :datetime_with_timezone)
-> 0.0007s
-- remove_column(:packages_package_files, :verification_state, :integer, {:default=>0, :limit=>2})
-> 0.0004s
== 20201112173532 AddVerificationStateToPackageFiles: reverted (0.0029s) ======
➜ gitlab git:(mk/add-verification-state-machine) ✗
Does this MR meet the acceptance criteria?
Conformity
-
Changelog entry -
Documentation (if required) -
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides -
Separation of EE specific content
Availability and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process.
Edited by Michael Kozono