<!--IssueSummary start--> <details> <summary> Everyone can contribute. [Help move this issue forward](https://handbook.gitlab.com/handbook/marketing/developer-relations/contributor-success/community-contributors-workflows/#contributor-links) while earning points, leveling up and collecting rewards. </summary> - [Close this issue](https://contributors.gitlab.com/manage-issue?action=close&projectId=278964&issueIid=343451) </details> <!--IssueSummary end--> Currently, GitLab file-based import sidekiq workers are not idempotent. This is not ideal, as multiple runs of such jobs can cause unexpected side effects, as duplicate object creation, uncaught exceptions, etc. As per the sidekiq guidelines, such jobs should ideally be idempotent and transactional https://gitlab.com/gitlab-org/gitlab/blob/master/doc/development/sidekiq_style_guide.md#L109-109 ## Considerations Consider the case of workers running more than once perhaps through deployments where sidekiq reliable fetch is invoked * somehow track the position of the import as it proceeds and only starts where it left off(in case of rerun) * position = filename and line number * record this position in Redis and only save the position again when the next line starts. * consider `upsert` if not too expensive of an operation instead of tracking the position. See - https://gitlab.com/gitlab-org/gitlab/-/issues/285111#note_685766705