Update project_daily_statistics synchronally
What does this MR do?
The project_daily_statistics queue can accumulate jobs
from time to time, considering how often the git_upload_pack
endpoint is called (taking up to 7 minutes to take the job
from the queue in a few cases) - so it can affect our SLOs:
See https://docs.gitlab.com/ee/development/sidekiq_style_guide.html#job-urgency
Turns out that even though this is a low-priority queue, it was seen that the work it does takes around 1.8s (99th percentile), and 0.8s (99th percentile) DB time, and way lower for p50 and p95, which doesn't explain involving Sidekiq (and Redis) in the work pipeline.
Source https://log.gprd.gitlab.net/goto/4fbc01a0b7aca3d774938fc365c11492
Here we move the processing out of Sidekiq and back to the web request behind a feature flag (and deprecate the worker).
gitlab-com/gl-infra/scalability#256 (closed)