Gitaly timing out in master
Summary
Update gitaly issue logged: gitlab-org/gitaly#2188 (closed)
Creating/accessing repos/files with the latest gitaly is failing in master and other new branches with timeout errors.
https://gitlab.com/gitlab-org/charts/gitlab/-/jobs/356461013
The following error can be seen in unicorn when trying to access an existing repo:
Gitlab::Git::CommandError (4:Deadline Exceeded):
lib/gitlab/git/wraps_gitaly_errors.rb:13:in `rescue in wrapped_gitaly_errors'
lib/gitlab/git/wraps_gitaly_errors.rb:6:in `wrapped_gitaly_errors'
lib/gitlab/metrics/instrumentation.rb:161:in `block in wrapped_gitaly_errors'
lib/gitlab/metrics/method_call.rb:36:in `measure'
lib/gitlab/metrics/instrumentation.rb:161:in `wrapped_gitaly_errors'
lib/gitlab/git/blob.rb:83:in `batch_lfs_pointers'
lib/extracts_path.rb:149:in `lfs_blob_ids'
app/controllers/application_controller.rb:533:in `block in allow_gitaly_ref_name_caching'
lib/gitlab/gitaly_client.rb:314:in `allow_ref_name_caching'
app/controllers/application_controller.rb:532:in `allow_gitaly_ref_name_caching'
ee/lib/gitlab/ip_address_state.rb:10:in `with'
ee/app/controllers/ee/application_controller.rb:43:in `set_current_ip_address'
lib/gitlab/session.rb:11:in `with_session'
app/controllers/application_controller.rb:450:in `set_session_storage'
lib/gitlab/i18n.rb:55:in `with_locale'
lib/gitlab/i18n.rb:61:in `with_user_locale'
app/controllers/application_controller.rb:444:in `set_locale'
lib/gitlab/middleware/rails_queue_duration.rb:27:in `call'
lib/gitlab/metrics/rack_middleware.rb:17:in `block in call'
lib/gitlab/metrics/transaction.rb:62:in `run'
lib/gitlab/metrics/rack_middleware.rb:17:in `call'
lib/gitlab/request_profiler/middleware.rb:17:in `call'
ee/lib/gitlab/jira/middleware.rb:19:in `call'
lib/gitlab/middleware/go.rb:20:in `call'
lib/gitlab/etag_caching/middleware.rb:13:in `call'
lib/gitlab/middleware/correlation_id.rb:16:in `block in call'
lib/gitlab/middleware/correlation_id.rb:15:in `call'
lib/gitlab/middleware/multipart.rb:117:in `call'
lib/gitlab/middleware/read_only/controller.rb:48:in `call'
lib/gitlab/middleware/read_only.rb:18:in `call'
lib/gitlab/middleware/basic_health_check.rb:25:in `call'
lib/gitlab/request_context.rb:32:in `call'
config/initializers/fix_local_cache_middleware.rb:9:in `call'
lib/gitlab/metrics/requests_rack_middleware.rb:49:in `call'
lib/gitlab/middleware/release_env.rb:12:in `call'
These aren't just limited to the tests, we get the same result when trying to view repos through the webui.
Edited by DJ Mountney