Geo: Make Job Artifacts replication use Self-Service Framework

What does this MR do and why?

Migrate Job Artifacts replication to the new Geo SSF framework. It also adds Job Artifacts verification.

Database Migrations

Up:

== 20211119111006 CreateJobArtifactStates: migrating ==========================
-- create_table(:job_artifact_states, {:id=>false})
-- quote_column_name(:verification_failure)
   -> 0.0000s
   -> 0.0091s
== 20211119111006 CreateJobArtifactStates: migrated (0.0092s) =================

Geo database:

== 20210111911002 PrepareJobArtifactRegistryForSsf: migrating ================
-- change_column_default(:job_artifact_registry, :retry_count, {:from=>nil, :to=>0})
   -> 0.0022s
-- add_column(:job_artifact_registry, :last_synced_at, :datetime_with_timezone)
   -> 0.0006s
-- add_column(:job_artifact_registry, :verified_at, :datetime_with_timezone)
   -> 0.0005s
-- add_column(:job_artifact_registry, :verification_started_at, :datetime_with_timezone)
   -> 0.0005s
-- add_column(:job_artifact_registry, :verification_retry_at, :datetime_with_timezone)
   -> 0.0005s
-- add_column(:job_artifact_registry, :state, :integer, {:null=>false, :limit=>2, :default=>0})
   -> 0.0009s
-- add_column(:job_artifact_registry, :verification_state, :integer, {:default=>0, :null=>false, :limit=>2})
   -> 0.0010s
-- add_column(:job_artifact_registry, :verification_retry_count, :integer, {:default=>0, :limit=>2, :null=>false})
   -> 0.0009s
-- add_column(:job_artifact_registry, :verification_checksum, :binary)
   -> 0.0006s
-- add_column(:job_artifact_registry, :verification_checksum_mismatched, :binary)
   -> 0.0006s
-- add_column(:job_artifact_registry, :checksum_mismatch, :boolean, {:default=>false, :null=>false})
   -> 0.0009s
-- add_column(:job_artifact_registry, :verification_failure, :string, {:limit=>255})
   -> 0.0006s
-- add_column(:job_artifact_registry, :last_sync_failure, :string, {:limit=>255})
   -> 0.0006s
-- transaction_open?()
   -> 0.0001s
-- index_exists?(:job_artifact_registry, :verification_retry_at, {:name=>:job_artifact_registry_failed_verification, :order=>"NULLS FIRST", :where=>"((state = 2) AND (verification_state = 3))", :algorithm=>:concurrently})
   -> 0.0024s
-- add_index(:job_artifact_registry, :verification_retry_at, {:name=>:job_artifact_registry_failed_verification, :order=>"NULLS FIRST", :where=>"((state = 2) AND (verification_state = 3))", :algorithm=>:concurrently})
   -> 0.0013s
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:job_artifact_registry, :verification_state, {:name=>:job_artifact_registry_needs_verification, :where=>"((state = 2)  AND (verification_state = ANY (ARRAY[0, 3])))", :algorithm=>:concurrently})
   -> 0.0029s
-- add_index(:job_artifact_registry, :verification_state, {:name=>:job_artifact_registry_needs_verification, :where=>"((state = 2)  AND (verification_state = ANY (ARRAY[0, 3])))", :algorithm=>:concurrently})
   -> 0.0013s
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:job_artifact_registry, :verified_at, {:name=>:job_artifact_registry_pending_verification, :order=>"NULLS FIRST", :where=>"((state = 2) AND (verification_state = 0))", :algorithm=>:concurrently})
   -> 0.0032s
-- add_index(:job_artifact_registry, :verified_at, {:name=>:job_artifact_registry_pending_verification, :order=>"NULLS FIRST", :where=>"((state = 2) AND (verification_state = 0))", :algorithm=>:concurrently})
   -> 0.0013s
== 20210111911002 PrepareJobArtifactRegistryForSsf: migrated (0.0283s) =======

== 20210111911110 FixStateColumnInJobArtifactRegistry: migrating =============
-- transaction_open?()
   -> 0.0000s
-- exec_query("SELECT COUNT(*) AS count FROM \"job_artifact_registry\" WHERE \"job_artifact_registry\".\"success\" = TRUE")
   -> 0.0010s
== 20210111911110 FixStateColumnInJobArtifactRegistry: migrated (0.0015s) ====

Down:

== 20211119111006 CreateJobArtifactStates: reverting ==========================
-- drop_table(:job_artifact_states)
   -> 0.0034s
== 20211119111006 CreateJobArtifactStates: reverted (0.0035s) =================

Geo database:

== 20210111911002 PrepareJobArtifactRegistryForSsf: reverting ================
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:job_artifact_registry, :verification_retry_at, {:name=>:job_artifact_registry_failed_verification, :algorithm=>:concurrently})
   -> 0.0058s
-- remove_index(:job_artifact_registry, {:name=>:job_artifact_registry_failed_verification, :algorithm=>:concurrently, :column=>:verification_retry_at})
   -> 0.0052s
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:job_artifact_registry, :verification_state, {:name=>:job_artifact_registry_needs_verification, :algorithm=>:concurrently})
   -> 0.0034s
-- remove_index(:job_artifact_registry, {:name=>:job_artifact_registry_needs_verification, :algorithm=>:concurrently, :column=>:verification_state})
   -> 0.0042s
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:job_artifact_registry, :verified_at, {:name=>:job_artifact_registry_pending_verification, :algorithm=>:concurrently})
   -> 0.0030s
-- remove_index(:job_artifact_registry, {:name=>:job_artifact_registry_pending_verification, :algorithm=>:concurrently, :column=>:verified_at})
   -> 0.0038s
== 20210111911002 PrepareJobArtifactRegistryForSsf: reverted (0.0301s) =======

== 20210111911110 FixStateColumnInJobArtifactRegistry: reverting =============
== 20210111911110 FixStateColumnInJobArtifactRegistry: reverted (0.0000s) ====

Job db:gitlabcom-database-testing - https://gitlab.com/gitlab-org/gitlab/-/jobs/2007833782

Database review notes

This MR is based on template changes from .gitlab/issue_templates/Geo Replicate a new blob type.md document. So basically, all the changes have been reviewed many times for the rest of the similar tables but let's see if we can spot anything else.

Screenshots or screen recordings

These are strongly recommended to assist reviewers and reduce the time to merge your change.

How to set up and validate locally

The only thing needed to enable the new feature is FF enabling:

1. Enable the invite modal

   Feature.enable(:geo_job_artifact_replication)

2. Check how Geo replicates Job Artifact in admin area. -->

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.

Related to #327400 (closed)