git push via HTTPS error: GRPC::Unavailable (14:CommitDelta: exit status 128)
zd: internal only https://gitlab.zendesk.com/agent/tickets/103920
Summary
When customer tries to push a repository to GitLab, a repository is cloned from github, it appears that the push times out. It appears that initially there was an issue with Unicorn
timing out.
==> /var/log/gitlab/unicorn/unicorn_stderr.log <==
W, [2018-09-18T13:55:54.617121 #8505] WARN -- : #<Unicorn::HttpServer:0x00007f81409d1d18>: worker (pid: 8505) exceeds memory limit (470616064.0 bytes > 441536114 bytes)
W, [2018-09-18T13:55:54.617193 #8505] WARN -- : Unicorn::WorkerKiller send SIGQUIT (pid: 8505) alive: 260 sec (trial 1)
I, [2018-09-18T13:55:54.645394 #8432] INFO -- : reaped #<Process::Status: pid 8505 exit 0> worker=1
I, [2018-09-18T13:55:54.743574 #9401] INFO -- : worker=1 ready
We have increased unicorn memory limits:
Increase unicorn memory:
unicorn['worker_memory_limit_min'] = "700 * 1 << 20"
unicorn['worker_memory_limit_max'] = "750 * 1 << 20"
And we increased unicorn worker timeout to 1200
, at this time unicorn timeout error went away.
After unicorn error went away we started to experience a new error which appears to be related to Gitaly:
$ git push --all origin
Enumerating objects: 17396, done.
Counting objects: 100% (17396/17396), done.
Delta compression using up to 8 threads.
Compressing objects: 100% (6080/6080), done.
Writing objects: 100% (17396/17396), 7.59 MiB | 7.95 MiB/s, done.
Total 17396 (delta 8543), reused 17396 (delta 8543)
error: RPC failed; curl 18 transfer closed with outstanding read data remaining
fatal: The remote end hung up unexpectedly
fatal: The remote end hung up unexpectedly
Everything up-to-date
From the log:
==> /var/log/gitlab/gitaly/current <==
2018-09-19_11:47:26.39587 time="2018-09-19T11:47:26Z" level=info msg="finished streaming call" error="rpc error: code = Canceled desc = rpc error: code = Unavailable desc = PostReceivePack: signal: terminated" grpc.code=Canceled grpc.method=PostReceivePack grpc.request.glRepository=project-125 grpc.request.repoPath=path/path.git grpc.request.repoStorage=default grpc.request.topLevelGroup=group grpc.service=gitaly.SmartHTTPService grpc.time_ms=411032 peer.address=@ span.kind=server system=grpc
2018-09-19_11:47:26.42782 time="2018-09-19T11:47:26Z" level=info msg="fatal: bad object 3c8c0dfe9a300acc4fa0964a32b487cddd4e26af" grpc.meta.client_name=gitlab-web grpc.method=CommitDelta grpc.request.glRepository=project-125 grpc.request.repoPath=path/path.git grpc.request.repoStorage=default grpc.request.topLevelGroup=gitlabtest grpc.service=gitaly.DiffService peer.address=@ span.kind=server system=grpc
2018-09-19_11:47:26.42802 time="2018-09-19T11:47:26Z" level=warning msg="finished streaming call" error="rpc error: code = Unavailable desc = CommitDelta: exit status 128" grpc.code=Unavailable grpc.meta.client_name=gitlab-web grpc.method=CommitDelta grpc.request.glRepository=project-125 grpc.request.repoPath=path/path.git grpc.request.repoStorage=default grpc.request.topLevelGroup=group grpc.service=gitaly.DiffService grpc.time_ms=116 peer.address=@ span.kind=server system=grpc
==> /var/log/gitlab/gitlab-rails/production.log <==
GRPC::Unavailable (14:CommitDelta: exit status 128):
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grpc-1.11.0-x86_64-linux/src/ruby/lib/grpc/generic/active_call.rb:31:in `check_status'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grpc-1.11.0-x86_64-linux/src/ruby/lib/grpc/generic/active_call.rb:180:in `attach_status_results_and_complete_call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grpc-1.11.0-x86_64-linux/src/ruby/lib/grpc/generic/active_call.rb:169:in `receive_and_check_status'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grpc-1.11.0-x86_64-linux/src/ruby/lib/grpc/generic/active_call.rb:328:in `block in each_remote_read_then_finish'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grpc-1.11.0-x86_64-linux/src/ruby/lib/grpc/generic/active_call.rb:325:in `loop'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grpc-1.11.0-x86_64-linux/src/ruby/lib/grpc/generic/active_call.rb:325:in `each_remote_read_then_finish'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/gitaly_client/commit_service.rb:75:in `each'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/gitaly_client/commit_service.rb:75:in `flat_map'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/gitaly_client/commit_service.rb:75:in `commit_deltas'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/git/commit.rb:251:in `deltas'
/opt/gitlab/embedded/service/gitlab-rails/app/models/commit.rb:440:in `raw_deltas'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/checks/commit_check.rb:19:in `validate'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:159:in `block in validate'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/method_call.rb:34:in `measure'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:159:in `validate'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/checks/change_access.rb:131:in `block (2 levels) in commits_check'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/checks/change_access.rb:130:in `each'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/checks/change_access.rb:130:in `block in commits_check'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/gitaly_client.rb:333:in `allow_n_plus_1_calls'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/checks/change_access.rb:129:in `commits_check'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:159:in `block in commits_check'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/method_call.rb:34:in `measure'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:159:in `commits_check'
/opt/gitlab/embedded/service/gitlab-rails/ee/lib/ee/gitlab/checks/change_access.rb:26:in `exec'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/git_access.rb:292:in `check_single_change_access'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:159:in `block in check_single_change_access'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/method_call.rb:34:in `measure'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:159:in `check_single_change_access'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/git_access.rb:272:in `block in check_change_access!'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/git_access.rb:267:in `each'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/git_access.rb:267:in `with_index'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/git_access.rb:267:in `check_change_access!'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:159:in `block in check_change_access!'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/method_call.rb:34:in `measure'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:159:in `check_change_access!'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/git_access.rb:255:in `check_push_access!'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:159:in `block in check_push_access!'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/method_call.rb:34:in `measure'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:159:in `check_push_access!'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/git_access.rb:68:in `check'
/opt/gitlab/embedded/service/gitlab-rails/ee/lib/ee/gitlab/git_access.rb:11:in `check'
/opt/gitlab/embedded/service/gitlab-rails/lib/api/internal.rb:53:in `block (2 levels) in <class:Internal>'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grape-1.0.3/lib/grape/endpoint.rb:57:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grape-1.0.3/lib/grape/endpoint.rb:57:in `block (2 levels) in generate_api_method'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/activesupport-4.2.10/lib/active_support/notifications.rb:166:in `instrument'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grape-1.0.3/lib/grape/endpoint.rb:56:in `block in generate_api_method'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grape-1.0.3/lib/grape/endpoint.rb:262:in `block in run'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/activesupport-4.2.10/lib/active_support/notifications.rb:166:in `instrument'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grape-1.0.3/lib/grape/endpoint.rb:243:in `run'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grape-1.0.3/lib/grape/endpoint.rb:313:in `block in build_stack'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grape-1.0.3/lib/grape/middleware/base.rb:31:in `call!'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grape-1.0.3/lib/grape/middleware/base.rb:24:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grape-1.0.3/lib/grape/middleware/base.rb:31:in `call!'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grape-1.0.3/lib/grape/middleware/base.rb:24:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/rack-oauth2-1.2.3/lib/rack/oauth2/server/resource.rb:20:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/rack-oauth2-1.2.3/lib/rack/oauth2/server/resource/bearer.rb:8:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grape-1.0.3/lib/grape/middleware/error.rb:37:in `block in call!'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grape-1.0.3/lib/grape/middleware/error.rb:36:in `catch'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grape-1.0.3/lib/grape/middleware/error.rb:36:in `call!'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grape-1.0.3/lib/grape/middleware/base.rb:24:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grape_logging-1.7.0/lib/grape_logging/middleware/request_logger.rb:60:in `block in call!'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grape_logging-1.7.0/lib/grape_logging/middleware/request_logger.rb:58:in `catch'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grape_logging-1.7.0/lib/grape_logging/middleware/request_logger.rb:58:in `call!'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grape-1.0.3/lib/grape/middleware/base.rb:24:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/rack-1.6.10/lib/rack/head.rb:13:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grape-1.0.3/lib/grape/endpoint.rb:227:in `call!'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grape-1.0.3/lib/grape/endpoint.rb:221:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grape-1.0.3/lib/grape/router/route.rb:72:in `exec'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grape-1.0.3/lib/grape/router.rb:121:in `process_route'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grape-1.0.3/lib/grape/router.rb:74:in `block in identity'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grape-1.0.3/lib/grape/router.rb:93:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grape-1.0.3/lib/grape/router.rb:72:in `identity'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grape-1.0.3/lib/grape/router.rb:57:in `block in call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grape-1.0.3/lib/grape/router.rb:137:in `with_optimization'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grape-1.0.3/lib/grape/router.rb:56:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grape-1.0.3/lib/grape/api.rb:119:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grape-1.0.3/lib/grape/api.rb:45:in `call!'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grape-1.0.3/lib/grape/api.rb:40:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/actionpack-4.2.10/lib/action_dispatch/routing/mapper.rb:51:in `serve'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/actionpack-4.2.10/lib/action_dispatch/journey/router.rb:43:in `block in serve'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/actionpack-4.2.10/lib/action_dispatch/journey/router.rb:30:in `each'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/actionpack-4.2.10/lib/action_dispatch/journey/router.rb:30:in `serve'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/actionpack-4.2.10/lib/action_dispatch/routing/route_set.rb:817:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/omniauth-1.8.1/lib/omniauth/strategy.rb:415:in `call_app!'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/omniauth-saml-1.10.0/lib/omniauth/strategies/saml.rb:86:in `other_phase'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/omniauth-1.8.1/lib/omniauth/strategy.rb:189:in `call!'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/omniauth-1.8.1/lib/omniauth/strategy.rb:168:in `call'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/multipart.rb:97:in `call'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/request_profiler/middleware.rb:14:in `call'
/opt/gitlab/embedded/service/gitlab-rails/ee/lib/gitlab/jira/middleware.rb:15:in `call'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/go.rb:17:in `call'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/etag_caching/middleware.rb:11:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/batch-loader-1.2.1/lib/batch_loader/middleware.rb:11:in `call'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/rails_queue_duration.rb:22:in `call'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/rack_middleware.rb:15:in `block in call'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/transaction.rb:53:in `run'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/rack_middleware.rb:15:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/rack-attack-4.4.1/lib/rack/attack.rb:107:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/warden-1.2.7/lib/warden/manager.rb:36:in `block in call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/warden-1.2.7/lib/warden/manager.rb:35:in `catch'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/warden-1.2.7/lib/warden/manager.rb:35:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/rack-cors-1.0.2/lib/rack/cors.rb:97:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/rack-1.6.10/lib/rack/etag.rb:24:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/rack-1.6.10/lib/rack/conditionalget.rb:38:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/rack-1.6.10/lib/rack/head.rb:13:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/actionpack-4.2.10/lib/action_dispatch/middleware/params_parser.rb:27:in `call'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/read_only/controller.rb:40:in `call'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/read_only.rb:16:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/actionpack-4.2.10/lib/action_dispatch/middleware/flash.rb:260:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/rack-1.6.10/lib/rack/session/abstract/id.rb:225:in `context'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/rack-1.6.10/lib/rack/session/abstract/id.rb:220:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/actionpack-4.2.10/lib/action_dispatch/middleware/cookies.rb:560:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/activerecord-4.2.10/lib/active_record/query_cache.rb:36:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/activerecord-4.2.10/lib/active_record/connection_adapters/abstract/connection_pool.rb:653:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/actionpack-4.2.10/lib/action_dispatch/middleware/callbacks.rb:29:in `block in call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:88:in `__run_callbacks__'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:778:in `_run_call_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:81:in `run_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/actionpack-4.2.10/lib/action_dispatch/middleware/callbacks.rb:27:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/actionpack-4.2.10/lib/action_dispatch/middleware/remote_ip.rb:78:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/actionpack-4.2.10/lib/action_dispatch/middleware/debug_exceptions.rb:17:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/actionpack-4.2.10/lib/action_dispatch/middleware/show_exceptions.rb:30:in `call'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/basic_health_check.rb:25:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/railties-4.2.10/lib/rails/rack/logger.rb:38:in `call_app'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/railties-4.2.10/lib/rails/rack/logger.rb:20:in `block in call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/activesupport-4.2.10/lib/active_support/tagged_logging.rb:68:in `block in tagged'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/activesupport-4.2.10/lib/active_support/tagged_logging.rb:26:in `tagged'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/activesupport-4.2.10/lib/active_support/tagged_logging.rb:68:in `tagged'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/railties-4.2.10/lib/rails/rack/logger.rb:20:in `call'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/request_context.rb:18:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/request_store-1.3.1/lib/request_store/middleware.rb:9:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/actionpack-4.2.10/lib/action_dispatch/middleware/request_id.rb:21:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/rack-1.6.10/lib/rack/methodoverride.rb:22:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/rack-1.6.10/lib/rack/runtime.rb:18:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/rack-1.6.10/lib/rack/lock.rb:17:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/rack-1.6.10/lib/rack/sendfile.rb:113:in `call'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/requests_rack_middleware.rb:27:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/sentry-raven-2.7.2/lib/raven/integrations/rack.rb:51:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/railties-4.2.10/lib/rails/engine.rb:518:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/railties-4.2.10/lib/rails/application.rb:165:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/railties-4.2.10/lib/rails/railtie.rb:194:in `public_send'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/railties-4.2.10/lib/rails/railtie.rb:194:in `method_missing'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/release_env.rb:10:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/rack-1.6.10/lib/rack/urlmap.rb:66:in `block in call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/rack-1.6.10/lib/rack/urlmap.rb:50:in `each'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/rack-1.6.10/lib/rack/urlmap.rb:50:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/unicorn-5.1.0/lib/unicorn/http_server.rb:562:in `process_client'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/unicorn-worker-killer-0.4.4/lib/unicorn/worker_killer.rb:52:in `process_client'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/unicorn-5.1.0/lib/unicorn/http_server.rb:658:in `worker_loop'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/unicorn-5.1.0/lib/unicorn/http_server.rb:508:in `spawn_missing_workers'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/unicorn-5.1.0/lib/unicorn/http_server.rb:132:in `start'
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/unicorn-5.1.0/bin/unicorn:126:in `<top (required)>'
/opt/gitlab/embedded/bin/unicorn:23:in `load'
/opt/gitlab/embedded/bin/unicorn:23:in `<main>'
Some more details:
- The size of the repository being pushed is about 15mb however it has huge amounts of branches and commits.
- Just to make sure we have checked repository push rules for a limit on the repos in the
admin/push_rule
and repo settings. - We can not test push via SSH as we have no access with SSH.
- the Same user is also able to push to other projects without issue.
- We checked
Feature.enabled? :gitaly_commit_deltas
wasfalse
. - Your Gitlab setup is Omnibus package installation on AWS.
- GitLab version:
11.2.3-ee (aadca99)
Steps to reproduce
I was not able to reproduce this in my local test environment.
What is the current bug behavior?
When pushing to self-hosted EE instance it fails with the mentioned Gitaly error.
Edited by Julius Kvedaras