Skip to content
GitLab
Next
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
  • scalability scalability
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Locked Files
  • Issues 270
    • Issues 270
    • List
    • Boards
    • Service Desk
    • Milestones
    • Iterations
    • Requirements
  • Merge requests 1
    • Merge requests 1
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • Code review
    • Insights
    • Issue
    • Repository
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Commits
  • Issue Boards
Collapse sidebar
  • GitLab.comGitLab.com
  • GitLab Infrastructure TeamGitLab Infrastructure Team
  • scalabilityscalability
  • Issues
  • #9
Closed
Open
Issue created Sep 10, 2019 by Andrew Newdigate@andrewnOwner

Include metadata in sidekiq jobs

primary issue: https://gitlab.com/gitlab-org/gitlab-ce/issues/65625


Spun out of production#1028 (closed)

When we are experiencing issues related to Sidekiq queues, it can be quite difficult to correlate the issue with a particular user, project or namespace.

It would be useful if, when emitting structured logs for Sidekiq events, we could include some meta-data such as the user who invoked the job, the project the job is related to etc.

Obviously not all jobs have these fields, but when they do, it would be useful to include these values.

Before

{
    "severity":"INFO",
    "time":"2019-08-05T14:16:27.894Z",
    "class":"NewNoteWorker",
    "args":["200144415"],
    "retry":3,
    "queue":"new_note",
    "jid":"f7bd388f608e5fd4de46c6c2",
    "created_at":"2019-08-05T11:17:58.171Z",
    "correlation_id":"TEN7oNvplx4",
    "enqueued_at":"2019-08-05T11:17:58.175Z",
    "pid":17779,
    "message":"NewNoteWorker JID-f7bd388f608e5fd4de46c6c2: done: 716.241 sec",
    "job_status":"done",
    "duration":716.241,
    "completed_at":"2019-08-05T14:16:27.894Z",
    "tag":"sidekiq",
    "environment":"gprd",
    "hostname":"sidekiq-asap-02-sv-gprd",
    "fqdn":"sidekiq-asap-02-sv-gprd.c.gitlab-production.internal"
}

After

{
    "severity":"INFO",
    "time":"2019-08-05T14:16:27.894Z",
    "class":"NewNoteWorker",
    "args":["200144415"],
    "retry":3,
    "queue":"new_note",
    "jid":"f7bd388f608e5fd4de46c6c2",
    "created_at":"2019-08-05T11:17:58.171Z",
    "correlation_id":"TEN7oNvplx4",
    "enqueued_at":"2019-08-05T11:17:58.175Z",
    "pid":17779,
    "message":"NewNoteWorker JID-f7bd388f608e5fd4de46c6c2: done: 716.241 sec",
    "job_status":"done",
    "duration":716.241,
    "completed_at":"2019-08-05T14:16:27.894Z",
    "tag":"sidekiq",
    "environment":"gprd",
    "hostname":"sidekiq-asap-02-sv-gprd",
    "fqdn":"sidekiq-asap-02-sv-gprd.c.gitlab-production.internal",
    "meta.user": "andrewn",
    "meta.gl_project_path": "andrewn/andrewn",
    "meta.gl_toplevel_namespace": "andrewn" 
}
Edited Dec 03, 2019 by Bob Van Landuyt
Assignee
Assign to
Time tracking