logging for project_services sidekiq failures
When a project_services
job fails it's very difficult right now to determine the root cause because we cannot easily track down the exactly what endpoint is failing. example:
{"severity":"WARN","time":"2018-08-21T13:02:13.120Z","message":"OpenSSL::SSL::SSLError: SSL_connect returned=1 errno=0 state=error: certificate verify failed","tag":"sidekiq","environment":"gprd","hostname":"sidekiq-asap-04-sv-gprd","fqdn":"sidekiq-asap-04-sv-gprd.c.gitlab-production.internal"}
This is a sidekiq exception from a webhook that is connecting to an endpoint with a bad certificate. We would like to make this much easier to pull out of structured logging so when these types of exceptions start appearing in sentry/logging the dependency can be identified without inspecting the project configuration.
Additional context in slack https://gitlab.slack.com/archives/C8HG8D9MY/p1534856896000100