Preserve Geo verification retry fields across resyncs

What does this MR do and why?

verification_retry_count on a Geo registry can never exceed 1, and checksum_mismatch is cleared before every re-verification. Two behaviours combine to cause it:

• A verification failure calls failed in before_verification_failed, so the registry resyncs. While the sync state is failed the row is excluded from verification_failed_batch and needs_verification, because available_verifiables is synced — so verification cannot retry until the resync completes.
• The resync's :start transition disables verification (Geo::VerifiableRegistry#before_started), and after_synced then fires verification_pending! from verification_disabled, which hit the branch that clears the verification failure fields.

So the counter oscillates 1 → 0 forever:

verification_state=verification_failed   sync_state=failed  checksum_mismatch=true   verification_retry_count=1
verification_state=verification_pending  sync_state=synced  checksum_mismatch=false  verification_retry_count=0
verification_state=verification_failed   sync_state=failed  checksum_mismatch=true   verification_retry_count=1
verification_state=verification_pending  sync_state=synced  checksum_mismatch=false  verification_retry_count=0

That defeats the progressive backoff of syncs-due-to-verification-failures that before_verification_failed documents, and makes the counter unusable as a "how many consecutive verification failures" signal.

This MR includes verification_disabled in the transition that preserves the retry fields, alongside verification_failed. With it, the counter accumulates as intended:

verification_state=verification_failed   sync_state=failed  checksum_mismatch=true  verification_retry_count=1
verification_state=verification_pending  sync_state=synced  checksum_mismatch=true  verification_retry_count=1
verification_state=verification_failed   sync_state=failed  checksum_mismatch=true  verification_retry_count=2
verification_state=verification_pending  sync_state=synced  checksum_mismatch=true  verification_retry_count=2
verification_state=verification_failed   sync_state=failed  checksum_mismatch=true  verification_retry_count=3
verification_state=verification_pending  sync_state=synced  checksum_mismatch=true  verification_retry_count=3

Preserving the fields for a row leaving verification_disabled for other reasons (for example, the primary had not verified the resource yet) is harmless: needs_verification matches verification_pending rows regardless of verification_retry_at, so nothing is delayed, and the fields are still cleared on any => verification_succeeded. The second new example covers that.

Where the regression came from

08492a70 disabled verification on the :start transition so that invalid verification_failed rows (with a NULL verification_failure) could no longer block resyncs — #562921 (closed). Before it, start! left the row in verification_failed, so after_synced took the verification_failed => verification_pending branch and the counter survived, which is what the backoff comment still assumes. That fix is kept here; only the clearing changes.

Why it was not caught

Specs that need a non-zero verification_retry_count set the columns directly rather than driving the state machines. The existing comment on .verification_failed_batch in the shared examples notes the same thing from the other direction — that synced + verification_failed "should not happen anymore … because of the automatic resync of verification failures".

The new examples drive the real state machines instead, so they run for every verifiable registry class.

How to set up and validate locally

# On a secondary
registry = Geo::ProjectRepositoryRegistry.find_by(project_id: <id>)
allow(registry).to receive(:ready_to_verify?).and_return(true) # or use a project whose primary verification succeeded

3.times do
  registry.verification_failed_with_message!('Checksum does not match the primary checksum')
  registry.start!    # the resync scheduled by the verification failure
  registry.synced!
  puts registry.reload.verification_retry_count
end

Before this MR that prints 1, 1, 1. After it, 1, 2, 3.

Automated:

bundle exec rspec ee/spec/models/geo/project_repository_registry_spec.rb -e "when a verification failure triggers a resync"

Verified the new examples fail on master (expected: 1, got: 0, expected: 2, got: 1, …) and pass with the fix. Full local runs: ee/spec/models/concerns/geo/ + ee/spec/models/geo/ → 7066 examples, 0 failures; Geo sync/registry services → 413 examples, 0 failures.

Related

Found while reviewing !246765 (merged), which uses checksum_mismatch + verification_retry_count >= N to detect persistent checksum mismatches. That detection cannot fire until this lands.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.