High volume of Sentry errors
On our Sentry instance, I noticed a high volume of traffic mostly coming from the gitlab-shared-runners
project (https://sentry.gitlab.net/gitlab/shared-gitlab-runners/) that's impacting our ability to log events in real-time. The top two events:
*errors.errorString: Problem while reading command output
File "/builds/gitlab-org/gitlab-runner/.gopath/src/gitlab.com/gitlab-org/gitlab-runner/helpers/sentry/log_hook.go", line 45, in Fire
File "/builds/gitlab-org/gitlab-runner/.gopath/src/gitlab.com/gitlab-org/gitlab-runner/vendor/github.com/sirupsen/logrus/hooks.go", line 28, in Fire
File "/builds/gitlab-org/gitlab-runner/.gopath/src/gitlab.com/gitlab-org/gitlab-runner/vendor/github.com/sirupsen/logrus/entry.go", line 246, in fireHooks
File "/builds/gitlab-org/gitlab-runner/.gopath/src/gitlab.com/gitlab-org/gitlab-runner/vendor/github.com/sirupsen/logrus/entry.go", line 224, in log
File "/builds/gitlab-org/gitlab-runner/.gopath/src/gitlab.com/gitlab-org/gitlab-runner/vendor/github.com/sirupsen/logrus/entry.go", line 268, in Log
File "/builds/gitlab-org/gitlab-runner/.gopath/src/gitlab.com/gitlab-org/gitlab-runner/vendor/github.com/sirupsen/logrus/entry.go", line 359, in Logln
File "/builds/gitlab-org/gitlab-runner/.gopath/src/gitlab.com/gitlab-org/gitlab-runner/vendor/github.com/sirupsen/logrus/entry.go", line 388, in Errorln
File "/builds/gitlab-org/gitlab-runner/.gopath/src/gitlab.com/gitlab-org/gitlab-runner/helpers/docker/machine_command.go", line 54, in watch
*errors.errorString: Failed to process runner
File "/builds/gitlab-org/gitlab-runner/.gopath/src/gitlab.com/gitlab-org/gitlab-runner/helpers/sentry/log_hook.go", line 45, in Fire
File "/builds/gitlab-org/gitlab-runner/.gopath/src/gitlab.com/gitlab-org/gitlab-runner/vendor/github.com/sirupsen/logrus/hooks.go", line 28, in Fire
File "/builds/gitlab-org/gitlab-runner/.gopath/src/gitlab.com/gitlab-org/gitlab-runner/vendor/github.com/sirupsen/logrus/entry.go", line 246, in fireHooks
File "/builds/gitlab-org/gitlab-runner/.gopath/src/gitlab.com/gitlab-org/gitlab-runner/vendor/github.com/sirupsen/logrus/entry.go", line 224, in log
File "/builds/gitlab-org/gitlab-runner/.gopath/src/gitlab.com/gitlab-org/gitlab-runner/vendor/github.com/sirupsen/logrus/entry.go", line 268, in Log
File "/builds/gitlab-org/gitlab-runner/.gopath/src/gitlab.com/gitlab-org/gitlab-runner/vendor/github.com/sirupsen/logrus/entry.go", line 297, in Error
File "/builds/gitlab-org/gitlab-runner/.gopath/src/gitlab.com/gitlab-org/gitlab-runner/commands/multi.go", line 279, in processRunners
@steveazz @tmaczukin Can we just squelch these events?