Skip to content
GitLab
Next
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
  • GitLab GitLab
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Locked Files
  • Issues 44,761
    • Issues 44,761
    • List
    • Boards
    • Service Desk
    • Milestones
    • Iterations
    • Requirements
  • Merge requests 1,329
    • Merge requests 1,329
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
    • Test Cases
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Container Registry
    • Infrastructure Registry
  • Monitor
    • Monitor
    • Metrics
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Code review
    • Insights
    • Issue
    • Repository
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • GitLab.orgGitLab.org
  • GitLabGitLab
  • Issues
  • #340629
Closed
Open
Issue created Sep 10, 2021 by Sean McGivern@smcgivernMaintainer

Hashed storage migration workers depend on their own queue size

Sidekiq worker routing allows administrators to redirect Sidekiq workers to different queues. The main use case for this is to put multiple workers in the same queue, which will result in lower Redis CPU usage (gitlab-com/gl-infra&469 (closed)).

The HashedStorage::RollbackerWorker and HashedStorage::ProjectRollbackWorker both use their own queue size via Gitlab::HashedStorage::Migrator#rollback_pending?.

The same goes for the HashedStorage::MigratorWorker and HashedStorage::ProjectMigrateWorker via Gitlab::HashedStorage::Migrator#migration_pending?.

This prevents these workers from sharing a queue with other workers. (Technically the two rollback workers could share a queue, as could the two migrator workers, but that's it.) This isn't a problem for GitLab.com, where we don't need to run any hashed storage migrations, but is a wrinkle for worker routing that we have to document.

Gitlab::BackgroundMigration.remaining handles this case correctly, and it should be easy enough to port to these workers.

Labelling as groupgitaly because that is mentioned in https://gitlab.com/gitlab-org/gitlab/blob/master/doc/administration/repository_storage_types.md.

Assignee
Assign to
Time tracking