Fix no_proxy not working when DNS rebinding protection enabled
What does this MR do and why?
When a URL request is made, the Gitlab::UrlBlocker
validates whether the request should be allowed to proceed and returns the URI and hostname that should be used to connect to the URL.
In GitLab 15.11,
!115343 (merged)
re-introduced the skipping of DNS rebinding checks if a proxy server is
enabled. It also carved out an exception for the no_proxy setting: if
the URL is configured to bypass a proxy, UrlBlocker
would allow the
DNS rebinding protection to go ahead. DNS rebinding protection
resolves the hostname to the IP address before the request is
initiated.
When the HTTP request were initiated, the Net::HTTP
request would
compare the resolved IP against the no_proxy
list. If the IP address
did not match any entries in no_proxy
, the request would be
erroneously be routed to the proxy server.
This change fixes the issue of no_proxy
being ignored by returning a
boolean, use_proxy
, in UrlBlocker
. If false
, this tells the
caller whether the proxy should be disabled, since the UrlBlocker
normally does the necessary checks to determine whether a proxy should
be used. This commit also refactors the return value to use a Struct.
Relates to #410636 (closed)
Screenshots or screen recordings
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
How to set up and validate locally
- Boot a
rails console
with an HTTP proxy configured:
https_proxy=http://localhost:9999 http_proxy=http://localhost:9999 no_proxy="google.com" bin/rails c
- Run:
Gitlab::HTTP.get('https://google.com/')
On master
, you should see something like:
[1] pry(main)> Gitlab::HTTP.get('https://google.com/')
Errno::ECONNREFUSED: Failed to open TCP connection to localhost:9999 (Connection refused - connect(2) for "localhost" port 9999)
from /Users/stanhu/.asdf/installs/ruby/3.0.5/lib/ruby/3.0.0/net/http.rb:987:in `initialize'
Caused by Errno::ECONNREFUSED: Connection refused - connect(2) for "localhost" port 9999
from /Users/stanhu/.asdf/installs/ruby/3.0.5/lib/ruby/3.0.0/net/http.rb:987:in `initialize'
On this branch, you should see a successful request:
[6] pry(main)> Gitlab::HTTP.get('https://google.com').first(100)
=> "<!doctype html><html itemscope=\"\" itemtype=\"http://schema.org/WebPage\" lang=\"en\"><head><meta content"
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.