test:update job fails
Job #281699937 failed for d7e18a81:
> Testing GDK attempt #2..
03:21:19 gitaly.1 | time="2019-08-28T03:21:18Z" level=warning msg="health check failed" error="rpc error: code = DeadlineExceeded desc = context deadline exceeded" worker.name=gitaly-ruby.1
03:21:20 gitaly.1 | time="2019-08-28T03:21:18Z" level=warning msg="health check failed" error="rpc error: code = DeadlineExceeded desc = context deadline exceeded" worker.name=gitaly-ruby.0
curl: (22) The requested URL returned error: 500 Internal Server Error
03:21:57 gitaly.1 | time="2019-08-28T03:21:47Z" level=warning msg="health check failed" error="rpc error: code = DeadlineExceeded desc = context deadline exceeded" worker.name=gitaly-ruby.0
03:21:58 gitaly.1 | time="2019-08-28T03:21:52Z" level=warning msg="health check failed" error="rpc error: code = DeadlineExceeded desc = context deadline exceeded" worker.name=gitaly-ruby.1
03:22:28 gitaly.1 | time="2019-08-28T03:22:22Z" level=warning msg="health check failed" error="rpc error: code = DeadlineExceeded desc = context deadline exceeded" worker.name=gitaly-ruby.1
03:22:29 gitaly.1 | time="2019-08-28T03:22:23Z" level=warning msg="health check failed" error="rpc error: code = DeadlineExceeded desc = context deadline exceeded" worker.name=gitaly-ruby.0
03:22:30 gitaly.1 | time="2019-08-28T03:22:30Z" level=info msg="removed from balancer due to sustained failing health checks" worker.name=gitaly-ruby.1 worker.pid=6665
03:22:56 gitaly.1 | time="2019-08-28T03:22:49Z" level=warning msg="health check failed" error="rpc error: code = DeadlineExceeded desc = context deadline exceeded" worker.name=gitaly-ruby.1
03:22:56 gitaly.1 | time="2019-08-28T03:22:49Z" level=warning msg="health check failed" error="rpc error: code = DeadlineExceeded desc = context deadline exceeded" worker.name=gitaly-ruby.0
03:23:24 gitaly.1 | time="2019-08-28T03:23:17Z" level=warning msg="health check failed" error="rpc error: code = DeadlineExceeded desc = context deadline exceeded" worker.name=gitaly-ruby.1
03:23:25 gitaly.1 | time="2019-08-28T03:23:18Z" level=warning msg="health check failed" error="rpc error: code = DeadlineExceeded desc = context deadline exceeded" worker.name=gitaly-ruby.0
03:23:49 gitaly.1 | time="2019-08-28T03:23:42Z" level=info msg="sending SIGTERM" worker.name=gitaly-ruby.1 worker.pid=6665
03:23:49 gitaly.1 | time="2019-08-28T03:23:47Z" level=warning msg="health check failed" error="rpc error: code = DeadlineExceeded desc = context deadline exceeded" worker.name=gitaly-ruby.1
03:23:49 gitaly.1 | time="2019-08-28T03:23:47Z" level=warning msg="health check failed" error="rpc error: code = DeadlineExceeded desc = context deadline exceeded" worker.name=gitaly-ruby.0
03:23:56 gitaly.1 | time="2019-08-28T03:23:52Z" level=info msg="I, [2019-08-28T03:06:38.325875 #6665] INFO -- sentry: ** [Raven] Raven 2.9.0 configured not to capture errors: DSN not set" supervisor.args="[bundle exec bin/ruby-cd /builds/gitlab-org/gitlab-development-kit/gitlab-development-kit /builds/gitlab-org/gitlab-development-kit/gitlab-development-kit/gitaly/ruby/bin/gitaly-ruby 6585 /tmp/gitaly-ruby705608324/socket.1]" supervisor.name=gitaly-ruby.1
03:23:58 gitaly.1 | time="2019-08-28T03:23:57Z" level=info msg="GRPC-RUBY: WARN: Using gitaly-proto 1.60.0" supervisor.args="[bundle exec bin/ruby-cd /builds/gitlab-org/gitlab-development-kit/gitlab-development-kit /builds/gitlab-org/gitlab-development-kit/gitlab-development-kit/gitaly/ruby/bin/gitaly-ruby 6585 /tmp/gitaly-ruby705608324/socket.1]" supervisor.name=gitaly-ruby.1
03:23:58 gitaly.1 | time="2019-08-28T03:23:58Z" level=info msg="GRPC-RUBY: WARN: failed call: #<GRPC::ActiveCall:0x0000564b396d1c78>" supervisor.args="[bundle exec bin/ruby-cd /builds/gitlab-org/gitlab-development-kit/gitlab-development-kit /builds/gitlab-org/gitlab-development-kit/gitlab-development-kit/gitaly/ruby/bin/gitaly-ruby 6585 /tmp/gitaly-ruby705608324/socket.1]" supervisor.name=gitaly-ruby.1
03:23:58 gitaly.1 | time="2019-08-28T03:23:58Z" level=info msg="call#run_batch failed somehow" supervisor.args="[bundle exec bin/ruby-cd /builds/gitlab-org/gitlab-development-kit/gitlab-development-kit /builds/gitlab-org/gitlab-development-kit/gitlab-development-kit/gitaly/ruby/bin/gitaly-ruby 6585 /tmp/gitaly-ruby705608324/socket.1]" supervisor.name=gitaly-ruby.1
03:23:58 gitaly.1 | time="2019-08-28T03:23:58Z" level=info msg="GRPC-RUBY: WARN: failed call: #<GRPC::ActiveCall:0x0000564b397187e0>" supervisor.args="[bundle exec bin/ruby-cd /builds/gitlab-org/gitlab-development-kit/gitlab-development-kit /builds/gitlab-org/gitlab-development-kit/gitlab-development-kit/gitaly/ruby/bin/gitaly-ruby 6585 /tmp/gitaly-ruby705608324/socket.1]" supervisor.name=gitaly-ruby.1
03:23:58 gitaly.1 | time="2019-08-28T03:23:58Z" level=info msg="call#run_batch failed somehow" supervisor.args="[bundle exec bin/ruby-cd /builds/gitlab-org/gitlab-development-kit/gitlab-development-kit /builds/gitlab-org/gitlab-development-kit/gitlab-development-kit/gitaly/ruby/bin/gitaly-ruby 6585 /tmp/gitaly-ruby705608324/socket.1]" supervisor.name=gitaly-ruby.1
This line seems relevant:
03:21:19 gitaly.1 | time="2019-08-28T03:21:18Z" level=warning msg="health check failed" error="rpc error: code = DeadlineExceeded desc = context deadline exceeded" worker.name=gitaly-ruby.1
Edited by Thong Kuah