Workers that cannot be easily migrated to one-queue-per-shard
In &469, we're moving from a one-worker-one-queue model to many-workers-one-queue. Some workers can't be easily migrated. That means we need to either exclude them from this work or fix them before migrating them to a shared queue.
Check their own queue size
These workers depend on checking their own queue size, which won't work in this model.
| Feature Category | Workers | Note |
|---|---|---|
| source_code_management |
UpdateAllMirrorsWorker / ProjectImportScheduleWorker
|
UpdateAllMirrorsWorker depends on the queue size for ProjectImportScheduleWorker, which is on catchall (#1064 (closed) / gitlab-org/gitlab#325496 (comment 550866012)) |
| source_code_management | ::HashedStorage::RollbackerWorker |
catchall but does not run on GitLab.com. Related to Gitlab::HashedStorage::Migrator: gitlab-org/gitlab#340629 (closed)
|
| source_code_management | ::HashedStorage::ProjectRollbackWorker |
catchall but does not run on GitLab.com. Related to Gitlab::HashedStorage::Migrator: gitlab-org/gitlab#340629 (closed)
|
Gitlab::SidekiqQueue |
Keep it as it is. We would need to make some noise around this when we make &194 or &447 available to self-managed installations who could also be using this. |
Previously: #1064 (closed), #995 (closed)
Jobs written to Redis without passing through the application
- EmailReceiverWorker
- ServiceDeskEmailReceiverWorker
The EmailReceiverWorker and ServiceDeskEmailReceiverWorker rely on our use of mail_room to take emails from an IMAP mailbox and push them into Sidekiq. mail_room does not use our application code directly, and it does not run our Sidekiq client middleware.
We already have these queue names in our application (to pass to mail_room) in Gitlab::MailRoom::ADDRESS_SPECIFIC_CONFIG, so we could use the worker routing there, but that is probably a task of its own rather than one we want to bundle in with other queues: #1263 (closed)