Allow enqueueing reactive caching worker with high or low urgency
What does this MR do and why?
Context
With !215784 (merged), the worker ReactiveCachingWorker's urgency was updated to high to address a problem raised in https://gitlab.com/gitlab-com/request-for-help/-/issues/3890.
The worker Median and 99th porcentile execution times met the criteria for this change, but certain implementations are quite far from this mark.
For example, the 99th porcentile is > 20s for executions called from MergeRequest:
Due to this, the groupcode review error budget was impacted negatevly (re !215784 (comment 2943181041)) alongside other groups (re error-budget-reports#63 (comment 2986380061))
Changes
This change introduces a new low-priority background worker for refreshing cached data that doesn't depend on external services.
Previously, all cache refresh jobs used the same priority level. Now, when the system needs to update cached data in the background (not when users are actively waiting).
The change is controlled by a feature flag low_urgency_reactive_caching_worker, to account for the delay between web-fleet and Sidekiq deployments. See !218033 (comment 3047955732) for more context
References
Closes #585506
Related to #584137
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.