Fail pull mirror when Sidekiq interruptions are exhausted
What?
Fail pull mirror when Sidekiq interruptions are exhausted
Why?
RepositoryUpdateMirrorWorker has retry disabled, so when Reliable Fetch
quarantines a job after repeated Sidekiq restarts nothing clears the
import state. It stays in started and the mirror is skipped by the
scheduler until StuckProjectImportJobsWorker sweeps it 24 hours later.
Retrying manually is a silent no-op during that window.
Mark the import state failed from the interruptions-exhausted callback so the normal retry backoff reschedules it. The jid check stops a late quarantine from clobbering a newer run of the same mirror.
References
Resolves #628725 (closed)
How to set up and validate locally
- Trigger the exhausted callback manually — In a Rails console on a GDK instance with a pull mirror configured, simulate the scenario:
project = Project.find(<id>)
import_state = project.import_state
import_state.update!(status: :started, jid: 'test-jid-123')
job = { 'args' => [project.id], 'jid' => 'test-jid-123' }
RepositoryUpdateMirrorWorker.interruptions_exhausted_block.call(job)
import_state.reload
# Expect: status == 'failed', last_error contains 'maximum number of interruptions'
# Expect: next_execution_timestamp is set (retry scheduled)- Verify JID guard — Repeat with a mismatched JID (
'jid' => 'wrong-jid') and confirmimport_state.statusremainsstarted. - Verify non-existent project — Call with a non-existent project ID and confirm no exception is raised.
- End-to-end (optional, harder to reproduce) — Configure a pull mirror pointing to an unreachable remote, then repeatedly restart Sidekiq while the job is running until Reliable Fetch quarantines it. Confirm the mirror transitions to
failedwith the expected error message rather than staying stuck instartedfor 24 hours. - Check retry scheduling — After step 1, verify
import_state.next_execution_timestampis in the future and the mirror appears in the scheduler queue.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Related to #628725 (closed)