url_blocker getaddrinfo should use a timeout
Blocked as Ruby currently doesn't support this well: https://bugs.ruby-lang.org/issues/17561 describes the problem.
Proposal
Add a 5-second timeout when we call Addrinfo.getaddrinfo
, like in gitlab-org/gitlab!51140 (closed).
Manual testing notes in gitlab-org/gitlab!51140 (comment 481320947).
Problem
In #449 (closed), we were seeing certain web requests hanging for over 5 minutes.
This problem appears to be resolved now, although one case still persists:
lib/gitlab/url_blocker.rb:111:in `getaddrinfo',
lib/gitlab/url_blocker.rb:111:in `get_address_info',
lib/gitlab/url_blocker.rb:48:in `validate!',
app/validators/addressable_url_validator.rb:83:in `validate_each',
ee/lib/gitlab/database/load_balancing/connection_proxy.rb:88:in `block in write_using_load_balancer',
ee/lib/gitlab/database/load_balancing/load_balancer.rb:84:in `block in read_write',
ee/lib/gitlab/database/load_balancing/load_balancer.rb:134:in `retry_with_backoff',
ee/lib/gitlab/database/load_balancing/load_balancer.rb:83:in `read_write',
ee/lib/gitlab/database/load_balancing/connection_proxy.rb:82:in `write_using_load_balancer',
ee/lib/gitlab/database/load_balancing/connection_proxy.rb:56:in `block (2 levels) in <class:ConnectionProxy>',
ee/lib/gitlab/database/load_balancing/connection_proxy.rb:88:in `block in write_using_load_balancer',
ee/lib/gitlab/database/load_balancing/load_balancer.rb:84:in `block in read_write',
ee/lib/gitlab/database/load_balancing/load_balancer.rb:134:in `retry_with_backoff',
ee/lib/gitlab/database/load_balancing/load_balancer.rb:83:in `read_write',
ee/lib/gitlab/database/load_balancing/connection_proxy.rb:82:in `write_using_load_balancer',
ee/lib/gitlab/database/load_balancing/connection_proxy.rb:56:in `block (2 levels) in <class:ConnectionProxy>',
lib/api/commit_statuses.rb:108:in `block (2 levels) in <class:CommitStatuses>',
ee/lib/gitlab/middleware/ip_restrictor.rb:14:in `block in call',
ee/lib/gitlab/ip_address_state.rb:10:in `with',
ee/lib/gitlab/middleware/ip_restrictor.rb:13:in `call',
ee/lib/omni_auth/strategies/group_saml.rb:41:in `other_phase',
lib/gitlab/metrics/elasticsearch_rack_middleware.rb:16:in `call',
lib/gitlab/middleware/rails_queue_duration.rb:33:in `call',
lib/gitlab/metrics/rack_middleware.rb:16:in `block in call',
lib/gitlab/metrics/transaction.rb:61:in `run',
lib/gitlab/metrics/rack_middleware.rb:16:in `call',
lib/gitlab/request_profiler/middleware.rb:17:in `call',
ee/lib/gitlab/database/load_balancing/rack_middleware.rb:39:in `call',
lib/gitlab/jira/middleware.rb:19:in `call',
lib/gitlab/middleware/go.rb:20:in `call',
lib/gitlab/etag_caching/middleware.rb:13:in `call',
lib/gitlab/middleware/multipart.rb:142:in `call',
lib/gitlab/middleware/read_only/controller.rb:51:in `call',
lib/gitlab/middleware/read_only.rb:18:in `call',
lib/gitlab/middleware/same_site_cookies.rb:27:in `call',
lib/gitlab/middleware/basic_health_check.rb:25:in `call',
lib/gitlab/middleware/handle_ip_spoof_attack_error.rb:25:in `call',
lib/gitlab/middleware/request_context.rb:23:in `call',
config/initializers/fix_local_cache_middleware.rb:9:in `call',
lib/gitlab/metrics/requests_rack_middleware.rb:60:in `call',
lib/gitlab/middleware/release_env.rb:12:in `call'
These calls sometimes take longer than 80s to timeout.
If it's possible, we should put a timeout on DNS resolution.
Edited by Andrew Newdigate