<!--IssueSummary start--> <details> <summary> Everyone can contribute. [Help move this issue forward](https://handbook.gitlab.com/handbook/marketing/developer-relations/contributor-success/community-contributors-workflows/#contributor-links) while earning points, leveling up and collecting rewards. </summary> - [Work on this issue](https://contributors.gitlab.com/manage-issue?action=work&projectId=278964&issueIid=435622) - [Close this issue](https://contributors.gitlab.com/manage-issue?action=close&projectId=278964&issueIid=435622) </details> <!--IssueSummary end--> This issue came out of this discussion: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/138437#note_1693841950). ### About `GithubImport::AdvanceStageWorker` is the only worker in the `GithubImport` namespace that doesn't share the `github_importer` [Sidekiq queue namespace](https://docs.gitlab.com/ee/development/sidekiq/#queue-namespaces). Our [docs explain the purpose of queue namespaces](https://docs.gitlab.com/ee/development/sidekiq/#queue-namespaces): > Defining a queue namespace for a worker makes it possible to start a Sidekiq process that automatically handles jobs for all workers in that namespace, without needing to explicitly list all their queue names. If, for example, all workers that are managed by sidekiq-cron use the cronjob queue namespace, we can spin up a Sidekiq process specifically for these kinds of scheduled jobs. If a new worker using the cronjob namespace is added later on, the Sidekiq process also picks up jobs for that worker (after having been restarted), without the need to change any configuration. ### Problem In https://gitlab.com/gitlab-org/gitlab/-/merge_requests/138437 we initially made a change to put `GithubImport::AdvanceStageWorker` in the `github_importer` queue namespace. It added a [post-migrate migration to move jobs](https://docs.gitlab.com/ee/development/sidekiq/compatibility_across_updates.html#renaming-queues) from the old queue to the new queue. It prompted a discussion about the time it might take between a prod deploy and the post-migration https://gitlab.com/gitlab-org/gitlab/-/merge_requests/138437#note_1693841950: > I'm trying to work out how long a job queued to `github_import_advance_stage` might be stuck until it's moved to `github_importer:github_import_advance_stage` and processed [...] Greater than a day can lead to a GitHub import failing because a `StuckProjectImportJobsWorker` cron worker will come and fail the import after 24 hours of no activity. We decided to go ahead with the other changes, and go back later and consider the task of changing namespaces for this worker. ### Questions for refinement 1. What is the risk of not having `AdvanceStageWorker` share the `github_importer` queue namespace with the other workers, now and in the future? 2. Can we ensure that we can migrate jobs in a way where running GitHub imports will not fail? Reading: - https://docs.gitlab.com/ee/development/sidekiq/#queue-namespaces - https://docs.gitlab.com/ee/development/sidekiq/compatibility_across_updates.html#renaming-queues ### Proposal 1. Migrate `GithubImport::AdvanceStageWorker` to `github_importer` Sidekiq queue - ideally with no possibility of failing in-progress GitHub imports. 1. Have the worker include `GithubImport::Queue` which will add it to the `github_importer` queue.