sometimes getting Rack::Timeout::RequestTimeoutException Gitlab 15.11
Hello,
we have a selfhosted gitlab instance and sometimes our jenkins ci/cd pipeline getting this error when pull the repository.
This is the error message in jenkins
2023-11-06 20:52:46 > git rev-parse --resolve-git-dir /srv/jenkins/caches/git-0aaaacf6a5576446cb4bb85c7dce8aaa/.git # timeout=60
2023-11-06 20:52:46 Setting origin to https://xxx.git
2023-11-06 20:52:46 > git config remote.origin.url https://xxx.git # timeout=60
2023-11-06 20:52:46 Fetching origin...
2023-11-06 20:52:46 Fetching upstream changes from origin
2023-11-06 20:52:46 > git --version # timeout=60
2023-11-06 20:52:46 > git --version # 'git version 2.30.2'
2023-11-06 20:52:46 > git config --get remote.origin.url # timeout=60
2023-11-06 20:52:46 using GIT_ASKPASS to set credentials gitlab xxx user for Jenkins
2023-11-06 20:52:46 > git fetch --tags --force --progress -- origin +refs/heads/*:refs/remotes/origin/* # timeout=60
2023-11-06 20:53:47 hudson.plugins.git.GitException: Command "git fetch --tags --force --progress -- origin +refs/heads/*:refs/remotes/origin/*" returned status code 128:
2023-11-06 20:53:47 stdout:
2023-11-06 20:53:47 stderr: error: RPC failed; HTTP 500 curl 22 The requested URL returned error: 500
2023-11-06 20:53:47 fatal: expected flush after ref listing`
and this is the error in gitlab production_json log
`"exception.class":"Rack::Timeout::RequestTimeoutException",
"exception.message":"Request ran for longer than 60000ms ",
"exception.backtrace":["lib/gitlab/auth/ldap/authentication.rb:29:in `login'",
"lib/gitlab/auth.rb:125:in `block (2 levels) in find_with_user_password'",
"lib/gitlab/auth.rb:124:in `each'",
"lib/gitlab/auth.rb:124:in `find'",
"lib/gitlab/auth.rb:124:in `block in find_with_user_password'",
"lib/gitlab/auth/unique_ips_limiter.rb:19:in `limit_user!'",
"lib/gitlab/auth.rb:101:in `find_with_user_password'",
"lib/gitlab/auth.rb:187:in `user_with_password_for_git'",
"lib/gitlab/auth.rb:72:in `find_for_git_client'",
"app/controllers/repositories/git_http_client_controller.rb:131:in `handle_basic_authentication'",
"app/controllers/repositories/git_http_client_controller.rb:49:in `authenticate_user'",
"ee/app/controllers/ee/repositories/git_http_controller.rb:71:in `authenticate_user'",
"ee/lib/gitlab/ip_address_state.rb:10:in `with'",
"ee/app/controllers/ee/application_controller.rb:45:in `set_current_ip_address'",
"lib/gitlab/metrics/elasticsearch_rack_middleware.rb:16:in `call'",
"lib/gitlab/middleware/memory_report.rb:13:in `call'",
"lib/gitlab/middleware/speedscope.rb:13:in `call'",
"lib/gitlab/database/load_balancing/rack_middleware.rb:23:in `call'",
"lib/gitlab/middleware/rails_queue_duration.rb:33:in `call'",
"lib/gitlab/metrics/rack_middleware.rb:16:in `block in call'",
"lib/gitlab/metrics/web_transaction.rb:46:in `run'",
"lib/gitlab/metrics/rack_middleware.rb:16:in `call'",
"lib/gitlab/jira/middleware.rb:19:in `call'",
"lib/gitlab/middleware/go.rb:20:in `call'",
"lib/gitlab/etag_caching/middleware.rb:21:in `call'",
"lib/gitlab/middleware/query_analyzer.rb:11:in `block in call'",
"lib/gitlab/database/query_analyzer.rb:37:in `within'",
"lib/gitlab/middleware/query_analyzer.rb:11:in `call'",
"lib/gitlab/middleware/multipart.rb:173:in `call'",
"lib/gitlab/middleware/read_only/controller.rb:50:in `call'",
"lib/gitlab/middleware/read_only.rb:18:in `call'",
"lib/gitlab/middleware/same_site_cookies.rb:27:in `call'",
"lib/gitlab/middleware/basic_health_check.rb:25:in `call'",
"lib/gitlab/middleware/handle_malformed_strings.rb:21:in `call'",
"lib/gitlab/middleware/handle_ip_spoof_attack_error.rb:25:in `call'",
"lib/gitlab/middleware/request_context.rb:21:in `call'",
"lib/gitlab/middleware/webhook_recursion_detection.rb:15:in `call'",
"config/initializers/fix_local_cache_middleware.rb:11:in `call'",
"lib/gitlab/middleware/compressed_json.rb:37:in `call'",
"lib/gitlab/middleware/rack_multipart_tempfile_factory.rb:19:in `call'",
"lib/gitlab/middleware/sidekiq_web_static.rb:20:in `call'",
"lib/gitlab/metrics/requests_rack_middleware.rb:79:in `call'",
"lib/gitlab/middleware/release_env.rb:13:in `call'"],
"db_duration_s":0.00387,
"view_duration_s":0.0,
"duration_s":59.99178}
do you know whats happen here? is it a problem with the ldap connection?!
Edited by Jason Plum