Add correlation_duration_s to Sidekiq and request logs

What does this MR do and why?

Sidekiq job logs and request logs now carry correlation_duration_s, the time in seconds since the root request or job of the call chain started. Chains of workers that keep running for hours, or loop back on themselves, can then be found in Kibana with a filter like json.correlation_duration_s > 3600.

A web request seeds root_started_at in Gitlab::ApplicationContext. Sidekiq jobs inherit it through Labkit the same way root_caller_id travels today, and jobs enqueued without a request, such as cron jobs, seed it at enqueue time. The structured logger derives the duration from it on the start and done lines. Decoding the timestamp in correlation_id would miss cron chains, because labkit-ruby still generates random hex IDs for them.

The new context key stays out of the Sidekiq queue deletion API, so the GraphQL schema does not change.

References

Screenshots or screen recordings

No UI changes.

How to set up and validate locally

  1. Push a commit to any project in GDK, so that PostReceive runs and enqueues further jobs.

  2. Read the Sidekiq log:

    tail -n 200 log/sidekiq.log | jq -c 'select(.correlation_duration_s != null) | {class, correlation_id, job_status, correlation_duration_s}'
  3. PostReceive and the jobs it enqueued share one correlation_id, and correlation_duration_s grows along the chain.

  4. Wait for any cron job to run. Its log lines also carry the field, measured from when the cron poller enqueued it.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Merge request reports

Loading
Loading