Skip to content

Direct Transfer - Change the logic to detect stale trackers

BulkImports::FinishBatchedPipelineWorker fails all BulkImports::BatchTracker associated to a tracker if the tracker is considered stale. To determine if a tracker is stale, the formula tracker.created_at < 4.hours.ago is used. And because all trackers are created when the entity migration starts, all batch migrations must be completed within 4 hours; otherwise, the migration fails.

Problem

Having a hardcoded period is problematic because the time could not be enough for large projects.

Proposed solution

As mentioned on #428110 (comment 1620770037), for batched migration, it's fine to hardcode a period because a batches process the same number of records no matter the size of the project.

So, to fix the described problem, we should remove the tracker.stale? condition and update FinishBatchedPipelineWorker worker to fail batches that didn't finish in a specific time.

Edited by Rodrigo Tomonari