Skip to content

Fix no_proxy not working when DNS rebinding protection enabled

Stan Hu requested to merge sh-fix-no-proxy-webhooks into master

What does this MR do and why?

When a URL request is made, the Gitlab::UrlBlocker validates whether the request should be allowed to proceed and returns the URI and hostname that should be used to connect to the URL.

In GitLab 15.11, !115343 (merged) re-introduced the skipping of DNS rebinding checks if a proxy server is enabled. It also carved out an exception for the no_proxy setting: if the URL is configured to bypass a proxy, UrlBlocker would allow the DNS rebinding protection to go ahead. DNS rebinding protection resolves the hostname to the IP address before the request is initiated.

When the HTTP request were initiated, the Net::HTTP request would compare the resolved IP against the no_proxy list. If the IP address did not match any entries in no_proxy, the request would be erroneously be routed to the proxy server.

This change fixes the issue of no_proxy being ignored by returning a boolean, use_proxy, in UrlBlocker. If false, this tells the caller whether the proxy should be disabled, since the UrlBlocker normally does the necessary checks to determine whether a proxy should be used. This commit also refactors the return value to use a Struct.

Relates to #410636 (closed)

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

How to set up and validate locally

  1. Boot a rails console with an HTTP proxy configured:
https_proxy=http://localhost:9999 http_proxy=http://localhost:9999 no_proxy="google.com" bin/rails c
  1. Run:
Gitlab::HTTP.get('https://google.com/')

On master, you should see something like:

[1] pry(main)> Gitlab::HTTP.get('https://google.com/')
Errno::ECONNREFUSED: Failed to open TCP connection to localhost:9999 (Connection refused - connect(2) for "localhost" port 9999)
from /Users/stanhu/.asdf/installs/ruby/3.0.5/lib/ruby/3.0.0/net/http.rb:987:in `initialize'
Caused by Errno::ECONNREFUSED: Connection refused - connect(2) for "localhost" port 9999
from /Users/stanhu/.asdf/installs/ruby/3.0.5/lib/ruby/3.0.0/net/http.rb:987:in `initialize'

On this branch, you should see a successful request:

[6] pry(main)> Gitlab::HTTP.get('https://google.com').first(100)
=> "<!doctype html><html itemscope=\"\" itemtype=\"http://schema.org/WebPage\" lang=\"en\"><head><meta content"

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Stan Hu

Merge request reports