URL is blocked: execution expired triggered for integrations when HTTP proxy is used and public DNS is not reachable
Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.
Summary
When some integrations (for example Slack) is used in an environment where:
- HTTP(s) proxy is used
- Public DNS is not reachable
The integrations stop working and Sidekiq prints
"exception.class": "Gitlab::HTTP_V2::BlockedUrlError",
"exception.message": "URL is blocked: execution expired",exceptions into exception_json.log.
To work-around this, it's enough to set the proper /etc/hosts records manually (does not need to be with the correct IPs)
Steps to reproduce
Set up environment to use any HTTPS proxy, configure local firewall to drop DNS packets, test e.g. Slack hook.
Example Project
n/a
What is the current bug behavior?
Integrations do not work and produce exceptions in log.
What is the expected correct behavior?
Integrations work as HTTPs proxy should be the one doing DNS lookup.
Relevant logs and/or screenshots
This is pretty much the same behaviour as described in #570525 (closed) , however that got closed since (presumably) the underlying customer issue got fixed.
Example exception is this:
{
"severity": "ERROR",
"time": "2025-10-15T05:32:54.044Z",
"correlation_id": "01K7K5NV0HK292YZZ9F1V931QF",
"meta.caller_id": "Integrations::ExecuteWorker",
"meta.remote_ip": "ip",
"meta.feature_category": "integrations",
"meta.user": "user",
"meta.user_id": 405,
"meta.project": "project",
"meta.root_namespace": "namespace",
"meta.client_id": "user/405",
"meta.organization_id": 1,
"meta.root_caller_id": "POST /api/:version/internal/post_receive",
"exception.class": "Gitlab::HTTP_V2::BlockedUrlError",
"exception.message": "URL is blocked: execution expired",
"exception.backtrace": [
"gems/gitlab-http/lib/gitlab/http_v2/new_connection_adapter.rb:65:in `rescue in validate_url_with_proxy!'",
"gems/gitlab-http/lib/gitlab/http_v2/new_connection_adapter.rb:62:in `validate_url_with_proxy!'",
"gems/gitlab-http/lib/gitlab/http_v2/new_connection_adapter.rb:39:in `connection'",
"httparty (0.23.1) lib/httparty/connection_adapter.rb:80:in `call'",
"httparty (0.23.1) lib/httparty/request.rb:201:in `http'",
"httparty (0.23.1) lib/httparty/request.rb:154:in `perform'",
"httparty (0.23.1) lib/httparty.rb:621:in `perform_request'",
"gems/gitlab-http/lib/gitlab/http_v2/client.rb:103:in `sync_perform_request'",
"gems/gitlab-http/lib/gitlab/http_v2/client.rb:68:in `perform_request'",
"httparty (0.23.1) lib/httparty.rb:547:in `post'",
"gems/gitlab-http/lib/gitlab/http_v2.rb:12:in `post'",
"lib/gitlab/http.rb:33:in `public_send'",
"lib/gitlab/http.rb:33:in `block (2 levels) in singleton class'",
"lib/integrations/clients/http.rb:28:in `post'",
"app/models/concerns/integrations/slack_mattermost_notifier.rb:37:in `post'",
"slack-messenger (2.3.6) lib/slack-messenger.rb:49:in `block in post'",
"slack-messenger (2.3.6) lib/slack-messenger.rb:47:in `map'",
"slack-messenger (2.3.6) lib/slack-messenger.rb:47:in `post'",
"slack-messenger (2.3.6) lib/slack-messenger.rb:37:in `ping'",
"app/models/concerns/integrations/slack_mattermost_notifier.rb:17:in `notify'",
"app/models/concerns/integrations/base/chat_notification.rb:137:in `execute'",
"app/workers/integrations/execute_worker.rb:25:in `perform'",
"vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'",
"vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'",
"lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'",
"lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'",
"lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'",
"lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'",
"lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'",
"lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:37:in `perform'",
"lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:11:in `call'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'",
"lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'",
"lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'",
"lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'",
"lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'",
"lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'",
"lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'",
"lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'",
"lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'",
"lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'",
"lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'",
"lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'",
"lib/gitlab/application_context.rb:176:in `block in use'",
"gitlab-labkit (0.40.0) lib/labkit/context.rb:36:in `with_context'",
"lib/gitlab/application_context.rb:176:in `use'",
"lib/gitlab/application_context.rb:98:in `with_context'",
"lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'",
"lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'",
"lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'",
"lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'",
"lib/gitlab/database/query_analyzer.rb:83:in `within'",
"lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'",
"lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'",
"lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'",
"lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:9:in `call'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'",
"lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'",
"lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'",
"lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `block in call'",
"lib/gitlab/sidekiq_middleware/server_metrics.rb:139:in `block in instrument'",
"lib/gitlab/metrics/background_transaction.rb:33:in `run'",
"lib/gitlab/sidekiq_middleware/server_metrics.rb:139:in `instrument'",
"lib/gitlab/sidekiq_middleware/server_metrics.rb:110:in `call'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'",
"lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'",
"gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'",
"gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'",
"lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'",
"gitlab-labkit (0.40.0) lib/labkit/middleware/sidekiq/server.rb:21:in `block in call'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'",
"gitlab-labkit (0.40.0) lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'",
"gitlab-labkit (0.40.0) lib/labkit/context.rb:36:in `with_context'",
"gitlab-labkit (0.40.0) lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'",
"gitlab-labkit (0.40.0) lib/labkit/middleware/sidekiq/server.rb:20:in `call'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'",
"lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'",
"lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'",
"lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'",
"lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'",
"lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'",
"lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'",
"lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'",
"marginalia (1.11.1) lib/marginalia/sidekiq_instrumentation.rb:9:in `call'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'",
"sentry-sidekiq (5.23.0) lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'",
"vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'",
"vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'",
"vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'",
"vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'",
"vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'",
"vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'",
"vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'",
"vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'",
"vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'",
"activesupport (7.1.5.2) lib/active_support/reloader.rb:77:in `block in wrap'",
"activesupport (7.1.5.2) lib/active_support/execution_wrapper.rb:92:in `wrap'",
"activesupport (7.1.5.2) lib/active_support/reloader.rb:74:in `wrap'",
"vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'",
"vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'",
"vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'",
"vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'",
"lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'",
"vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'",
"vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'",
"vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'",
"vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'",
"vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'",
"vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'",
"vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'",
"vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'",
"vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'",
"vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'",
"vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'",
"vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'",
"vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'",
"vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"
],
"exception.cause_class": "Gitlab::HTTP_V2::UrlBlocker::BlockedUrlError",
"user.username": "user",
"tags.queue": "default",
"tags.jid": "672f48c32de495524e628160",
"tags.program": "sidekiq",
"tags.locale": "en",
"tags.feature_category": "integrations",
"tags.correlation_id": "01K7K5NV0HK292YZZ9F1V931QF"
}Which is related to the gitlab-http gem and more specifically to the Gitlab::HTTP_V2::UrlBlocker.validate_resolved_uri behaviour. The current implementation counts with the option the DNS records are not resolvable when proxy is used:
begin
address_info = get_address_info(uri)
rescue SocketError
proxy_in_use = uri_under_proxy_setting?(uri, nil)
unless enforce_address_info_retrievable?(uri,
dns_rebind_protection,
deny_all_requests_except_allowed,
outbound_local_requests_allowlist)
return Result.new(uri, nil, proxy_in_use)
end
However that excepts SocketError is raised. But when DNS request timeouts (the environment I encountered this in uses internal DNS which does not have access towards internet, so DNS lookup towards e.g. slack.com just timeouts), which raises Timeout::Error:
irb(main):017> Gitlab::HTTP_V2::UrlBlocker::validate_url_with_proxy!("https://slack.com", schemes: ["https"], ports: [443])
gems/gitlab-http/lib/gitlab/http_v2/url_blocker.rb:225:in `rescue in get_address_info': execution expired (Gitlab::HTTP_V2::UrlBlocker::BlockedUrlError)
from gems/gitlab-http/lib/gitlab/http_v2/url_blocker.rb:218:in `get_address_info'
from gems/gitlab-http/lib/gitlab/http_v2/url_blocker.rb:124:in `validate_resolved_uri'
from gems/gitlab-http/lib/gitlab/http_v2/url_blocker.rb:88:in `validate_url_with_proxy!'
from (irb):17:in `<main>'
from railties (7.1.5.2) lib/rails/commands/console/console_command.rb:78:in `start'
from railties (7.1.5.2) lib/rails/commands/console/console_command.rb:16:in `start'
from railties (7.1.5.2) lib/rails/commands/console/console_command.rb:106:in `perform'
from thor (1.3.1) lib/thor/command.rb:28:in `run'
from thor (1.3.1) lib/thor/invocation.rb:127:in `invoke_command'
from railties (7.1.5.2) lib/rails/command/base.rb:178:in `invoke_command'
from thor (1.3.1) lib/thor.rb:527:in `dispatch'
from railties (7.1.5.2) lib/rails/command/base.rb:73:in `perform'
from railties (7.1.5.2) lib/rails/command.rb:71:in `block in invoke'
from railties (7.1.5.2) lib/rails/command.rb:149:in `with_argv'
from railties (7.1.5.2) lib/rails/command.rb:69:in `invoke'
from railties (7.1.5.2) lib/rails/commands.rb:18:in `<top (required)>'
... 3 levels...
/opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/timeout-0.4.3/lib/timeout.rb:40:in `rescue in handle_timeout': execution expired (Timeout::Error)
from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/timeout-0.4.3/lib/timeout.rb:37:in `handle_timeout'
from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/timeout-0.4.3/lib/timeout.rb:194:in `timeout'
from /opt/gitlab/embedded/service/gitlab-rails/gems/gitlab-http/lib/gitlab/http_v2/url_blocker.rb:219:in `get_address_info'
from /opt/gitlab/embedded/service/gitlab-rails/gems/gitlab-http/lib/gitlab/http_v2/url_blocker.rb:124:in `validate_resolved_uri'
from /opt/gitlab/embedded/service/gitlab-rails/gems/gitlab-http/lib/gitlab/http_v2/url_blocker.rb:88:in `validate_url_with_proxy!'
from (irb):17:in `<main>'
from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/irb-1.15.1/lib/irb/workspace.rb:101:in `eval'
from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/irb-1.15.1/lib/irb/workspace.rb:101:in `evaluate'
from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/irb-1.15.1/lib/irb/context.rb:591:in `evaluate_expression'
from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/irb-1.15.1/lib/irb/context.rb:557:in `evaluate'
from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/irb-1.15.1/lib/irb.rb:201:in `block (2 levels) in eval_input'
from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/irb-1.15.1/lib/irb.rb:512:in `signal_status'
from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/irb-1.15.1/lib/irb.rb:193:in `block in eval_input'
from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/irb-1.15.1/lib/irb.rb:272:in `block in each_top_level_statement'
from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/irb-1.15.1/lib/irb.rb:269:in `loop'
from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/irb-1.15.1/lib/irb.rb:269:in `each_top_level_statement'
... 20 levels...
/opt/gitlab/embedded/service/gitlab-rails/gems/gitlab-http/lib/gitlab/http_v2/url_blocker.rb:220:in `getaddrinfo': execution expired (Timeout::ExitException)
from /opt/gitlab/embedded/service/gitlab-rails/gems/gitlab-http/lib/gitlab/http_v2/url_blocker.rb:220:in `block in get_address_info'
from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/timeout-0.4.3/lib/timeout.rb:185:in `block in timeout'
from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/timeout-0.4.3/lib/timeout.rb:38:in `handle_timeout'
from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/timeout-0.4.3/lib/timeout.rb:194:in `timeout'
from /opt/gitlab/embedded/service/gitlab-rails/gems/gitlab-http/lib/gitlab/http_v2/url_blocker.rb:219:in `get_address_info'
from /opt/gitlab/embedded/service/gitlab-rails/gems/gitlab-http/lib/gitlab/http_v2/url_blocker.rb:124:in `validate_resolved_uri'
from /opt/gitlab/embedded/service/gitlab-rails/gems/gitlab-http/lib/gitlab/http_v2/url_blocker.rb:88:in `validate_url_with_proxy!'
from (irb):17:in `<main>'
from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/irb-1.15.1/lib/irb/workspace.rb:101:in `eval'
from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/irb-1.15.1/lib/irb/workspace.rb:101:in `evaluate'
from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/irb-1.15.1/lib/irb/context.rb:591:in `evaluate_expression'
from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/irb-1.15.1/lib/irb/context.rb:557:in `evaluate'
from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/irb-1.15.1/lib/irb.rb:201:in `block (2 levels) in eval_input'
from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/irb-1.15.1/lib/irb.rb:512:in `signal_status'
from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/irb-1.15.1/lib/irb.rb:193:in `block in eval_input'
from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/irb-1.15.1/lib/irb.rb:272:in `block in each_top_level_statement'
... 22 levels...
So the exception is not handled and the whole validation fails.
I have not experienced that before, but then I wasn't checking the exception_json.log on regular basis, so no idea when this was actually introduced (whether it's related to Limit HTTP response size commits that changed usage of Gitlab::HTTP to Clients::HTTP).
https://gitlab.com/gitlab-org/gitlab/-/blob/master/gems/gitlab-http/lib/gitlab/http_v2/url_blocker.rb#L114 for reference of the actual code
Output of checks
Results of GitLab environment info
Expand for output related to GitLab environment info
System information System: Current User: git Using RVM: no Ruby Version: 3.2.8 Gem Version: 3.7.1 Bundler Version:2.7.1 Rake Version: 13.0.6 Redis Version: 7.2.10 Sidekiq Version:7.3.9 Go Version: unknown GitLab information Version: 18.4.2 Revision: 635bee939f2 Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: PostgreSQL DB Version: 16.8 URL: https://host HTTP Clone URL: https://host/some-group/some-project.git SSH Clone URL: git@host:some-group/some-project.git Using LDAP: no Using Omniauth: yes Omniauth Providers: saml GitLab Shell Version: 14.45.2 Repository storages: - default: tcp://ip:8075 GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell Gitaly - default Address: tcp://ip:8075 - default Version: 18.4.2 - default Git Version: 2.50.1
Results of GitLab application Check
Expand for output related to the GitLab application check
Checking GitLab subtasks ...Checking GitLab Shell ...
GitLab Shell: ... GitLab Shell version >= 14.45.2 ? ... OK (14.45.2) Running /opt/gitlab/embedded/service/gitlab-shell/bin/gitlab-shell-check Internal API available: OK Redis available via internal API: OK gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Gitaly ...
Gitaly: ... default ... OK cluster ... OK
Checking Gitaly ... Finished
Checking Sidekiq ...
Sidekiq: ... Running? ... yes Number of Sidekiq processes (cluster/worker) ... 1/1
Checking Sidekiq ... Finished
Checking Incoming Email ...
Incoming Email: ... Reply by email is disabled in config/gitlab.yml
Checking Incoming Email ... Finished
Checking LDAP ...
LDAP: ... LDAP is disabled in config/gitlab.yml
Checking LDAP ... Finished
Checking GitLab App ...
Database config exists? ... yes Tables are truncated? ... skipped All migrations up? ... yes Database contains orphaned GroupMembers? ... no GitLab config exists? ... yes GitLab config up to date? ... yes Cable config exists? ... yes Resque config exists? ... yes Log directory writable? ... yes Tmp directory writable? ... yes Uploads directory exists? ... yes Uploads directory has correct permissions? ... yes Uploads directory tmp has correct permissions? ... skipped (no tmp uploads folder yet) Systemd unit files or init script exist? ... skipped (omnibus-gitlab has neither init script nor systemd units) Systemd unit files or init script up-to-date? ... skipped (omnibus-gitlab has neither init script nor systemd units) Projects have namespace: ... ... yes Redis version >= 6.2.14? ... yes Ruby version >= 3.0.6 ? ... yes (3.2.8) Git user has default SSH configuration? ... yes Is authorized keys file accessible? ... skipped (authorized keys not enabled) GitLab configured to store new projects in hashed storage? ... yes All projects are in hashed storage? ... yes
Checking GitLab App ... Finished
Checking GitLab subtasks ... Finished
Possible fixes
I would suggest modifying the logic so it expects these errors as well and I'm willing to prepare MR for it, but then I'm not sure on the correct approach to resolve this. The relevant get_address_info wraps the timeout exception into BlockedUrlError, which can then be also different then the one I'm encountering.
def get_address_info(uri)
Timeout.timeout(GETADDRINFO_TIMEOUT_SECONDS) do
Addrinfo.getaddrinfo(uri.hostname, get_port(uri), nil, :STREAM).map do |addr|
addr.ipv6_v4mapped? ? addr.ipv6_to_ipv4 : addr
end
end
rescue Timeout::Error => e
raise Gitlab::HTTP_V2::UrlBlocker::BlockedUrlError, e.message
rescue ArgumentError => e
# Addrinfo.getaddrinfo errors if the domain exceeds 1024 characters.
raise unless e.message.include?('hostname too long')
raise BlockedUrlError, "Host is too long (maximum is 1024 characters)"
endI'm not very fond of checking the exception message (since that can obviously easily change), but also I'm not sure modifying the behaviour of get_address_info would not break something else - also the GETADDRINFO_TIMEOUT_SECONDS is 15 seconds, which I'm not sure if isn't too much for these proxy checks, but then that might be different case.
I would suggest either creating new get_address_info function for these proxy checks that would not wrap the error, or reversing the actual validate_resolved_uri logic to first check for uri_under_proxy_setting? first and skipping the DNS resolve altogether.
Patch release information for backports
If the bug fix needs to be backported in a patch release to a version under the maintenance policy, please follow the steps on the patch release runbook for GitLab engineers.
Refer to the internal "Release Information" dashboard for information about the next patch release, including the targeted versions, expected release date, and current status.
High-severity bug remediation
To remediate high-severity issues requiring an internal release for single-tenant SaaS instances, refer to the internal release process for engineers.
Edited by 🤖 GitLab Bot 🤖