## Problem The `default` queue (weight 3) is a catch-all for jobs without explicit queue assignments. This creates several issues: 1. **Unclear intent**: Jobs in `default` queue have no documented priority rationale 2. **Mixed priorities**: May contain both important and trivial operations 3. **Discourages explicit assignment**: Developers may rely on default instead of choosing appropriate queue 4. **Difficult to optimize**: Can't tune performance without knowing what jobs are in the queue ## Current State The `default` queue has weight 3 (medium-high priority), which may be too high for a catch-all queue. **Unknown**: - What jobs currently use the `default` queue? - Are they all appropriate for weight 3? - Should some be moved to more specific queues? ## Proposal Audit all jobs using the `default` queue and migrate them to appropriate specialized queues. ### Phase 1: Discovery 1. **Identify all jobs in default queue**: ```ruby # Search codebase for jobs without explicit queue_as # Check Sidekiq metrics for jobs in default queue # Review recent job executions ``` 2. **Categorize jobs**: - Customer-facing operations → Move to appropriate high-priority queue - Internal operations → Move to appropriate medium-priority queue - Maintenance tasks → Move to low-priority queue - Truly generic → Keep in `default` but document why 3. **Document findings**: - List all jobs currently using `default` - Proposed queue for each job - Rationale for the assignment ### Phase 2: Migration 1. **Create missing specialized queues** (if needed): - Consider queues identified in other issues (#14269, #14271, etc.) 2. **Update job classes**: ```ruby # Before class SomeJob < ApplicationJob # No queue_as specified, uses :default end # After class SomeJob < ApplicationJob queue_as :appropriate_queue feature_category :relevant_category end ``` 3. **Update tests**: - Verify jobs are enqueued to correct queue - Update any tests that check queue names ### Phase 3: Reduce Default Queue Priority After migration, lower the `default` queue weight to discourage its use: ```yaml # config/sidekiq.yml :queues: # ... other queues ... - [default, 2] # Reduced from 3 ``` This encourages developers to explicitly choose appropriate queues for new jobs. ### Phase 4: Establish Guidelines Document when it's acceptable to use `default` queue: **Acceptable uses**: - Truly one-off administrative tasks - Jobs that genuinely don't fit any specialized queue - Temporary jobs during development (must be moved before production) **Not acceptable**: - Customer-facing operations - Revenue-impacting jobs - Jobs that fit an existing specialized queue - Long-running or resource-intensive jobs ## Implementation Steps 1. **Run audit** (1-2 days): - Query Sidekiq for jobs in default queue - Search codebase for jobs without `queue_as` - Review with team to categorize jobs 2. **Create migration plan** (1 day): - List all jobs to migrate - Identify target queues - Note any new queues needed 3. **Execute migration** (3-5 days): - Update job classes - Update tests - Deploy to staging - Verify correct queue usage 4. **Lower default weight** (after migration): - Update `config/sidekiq.yml` - Monitor for any issues - Document the change 5. **Add linting** (optional): - Consider adding a linter to flag jobs without explicit `queue_as` - Add to CI pipeline ## Success Criteria - All jobs have explicit queue assignments - `default` queue only contains truly generic jobs - `default` queue weight reflects its catch-all nature (weight 2) - Documentation exists for when to use `default` queue - No customer-facing or critical jobs in `default` queue ## Related - Parent epic: gitlab-org&19587 - Blocks: #14268 (weight granularity) - Related: All other queue optimization issues