Skip to content

Change Sidekiq log format to json

Priyan Sureshbabu requested to merge psureshbabu-master-patch-64401 into master

What does this MR do?

Currently, the Sidekiq pod logs provide minimal details as opposed to more detailed and structured json log in Omnibus.

Pod log sample:

2022-02-21T01:25:29.487Z 12 TID-5xdaw0 Namespaces::ScheduleAggregationWorker JID-94a9ff6f6d3849b1316766eb INFO: start
2022-02-21T01:25:29.489Z 12 TID-5xdaw0 Namespaces::ScheduleAggregationWorker JID-94a9ff6f6d3849b1316766eb INFO: arguments: [1101]
2022-02-21T01:25:29.500Z 12 TID-5xdaw0 Namespaces::ScheduleAggregationWorker JID-94a9ff6f6d3849b1316766eb INFO: done: 0.013 sec

Omnibus json log sample:

{"severity":"INFO","time":"2022-02-23T04:40:25.102Z","retry":3,"queue":"update_namespace_statistics:namespaces_schedule_aggregation","version":0,"queue_namespace":"update_namespace_statistics","class":"Namespaces::ScheduleAggregationWorker","args":["1"],"jid":"16e52447ecd70c911ae42593","created_at":"2022-02-23T04:40:25.070Z","meta.project":"root/mirror","meta.root_namespace":"root","meta.caller_id":"Projects::GitGarbageCollectWorker","meta.feature_category":"source_code_management","meta.client_id":"ip/","correlation_id":"388cec03cc498305af7e164965a2352f","worker_data_consistency":"always","idempotency_key":"resque:gitlab:duplicate:update_namespace_statistics:namespaces_schedule_aggregation:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2022-02-23T04:40:25.072Z","job_size_bytes":3,"pid":3413,"message":"Namespaces::ScheduleAggregationWorker JID-16e52447ecd70c911ae42593: start","job_status":"start","scheduling_latency_s":0.029956}

If you compare both the logs, the default pod log misses out a lot of useful information like resource consumption, caller details and different durations. It's very difficult to troubleshoot certain issues without those data, especially when trying to find out the root cause of a past(before we switch to json) issue.

This MR changes the default format to json.

Related issues

Checklist

See Definition of done.

For anything in this list which will not be completed, please provide a reason in the MR discussion.

Required

  • Merge Request Title and Description are up to date, accurate, and descriptive
  • MR targeting the appropriate branch
  • MR has a green pipeline on GitLab.com

Expected (please provide an explanation if not completing)

  • Test plan indicating conditions for success has been posted and passes
  • Documentation created/updated
  • Tests added
  • Integration tests added to GitLab QA
  • Equivalent MR/issue for omnibus-gitlab opened

Merge request reports