Add option to configure http-proxy for Pages

What does this MR do?

• Enables the ability to configure http-proxy more flexibly, easily toggle vs the limited check for external IP
• Doing so provides the ability to forcibly enable this when users have externalIP configured

Changelog: feature

Related issues

Closes: #2965 (closed)

Checklist

See Definition of done.

For anything in this list which will not be completed, please provide a reason in the MR discussion.

Testing

First, check for the use of listen-proxy having been enabled vs the use of listen-http in the Pages ConfigMap:

$ kubectl -n n2270 exec -i gitlab-gitlab-pages-6c69c9f5b5-6tskh -c gitlab-pages -- cat /etc/gitlab-pages/config
listen-proxy=0.0.0.0:8090
listen-https-proxyv2=0.0.0.0:8091
root-cert=/etc/gitlab-secrets/pages/pages.mnielsen.k8s-ft.win.crt
root-key=/etc/gitlab-secrets/pages/pages.mnielsen.k8s-ft.win.key
pages-domain=pages.mnielsen.k8s-ft.win
pages-root=/srv/gitlab-pages
log-format=json
log-verbose=true
redirect-http=false
use-http2=true
insecure-ciphers=false
artifacts-server=http://gitlab-webservice-default.n2270.svc:8181/api/v4
artifacts-server-timeout=10
gitlab-server=https://gitlab.mnielsen.k8s-ft.win
internal-gitlab-server=http://gitlab-webservice-default.n2270.svc:8181
api-secret-key=/etc/gitlab-secrets/pages/secret
domain-config-source=gitlab
metrics-address=:9235
pages-status=/-/readiness

For functional testing:

1. Deploy Pages

global:
  pages:
    enabled: true
    externalHttp:
      - 1.2.3.4 (external IP for the Pages Service)
    externalHttps:
      - 1.2.3.4 (external IP for the Pages Service)

gitlab:
  gitlab-pages:
    statusUri: "/-/readiness"
    useHTTPProxy: true

Create a sample Pages project from one of the templates and ensure the site is accessible.

2. Deploy HAProxy

We used a local HAProxy Docker container

global
  maxconn 4096
  log stdout local0 debug

defaults
  log global
  mode http
  retries 3
  option httplog

frontend test
  bind 0.0.0.0:80
  default_backend pages
  http-request set-header X-Forwarded-For %[src]

backend pages
  server s1 1.2.3.4:80 # replace 1.2.3.4 with external IP of Pages Service
  option httpchk GET /-/readiness

$ docker run -p 80:80 -v $(pwd)/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg haproxy:latest

3. Hit Pages through HAProxy

$ curl -Iv http://root.pages.mnielsen.k8s-ft.win/pages-demo --resolve root.pages.mnielsen.k8s-ft.win:80:127.0.0.1
HTTP/1.1 302 Found
* Added root.pages.mnielsen.k8s-ft.win:80:0.0.0.0 to DNS cache
* Hostname root.pages.mnielsen.k8s-ft.win was found in DNS cache
*   Trying 0.0.0.0:80...
* Connected to root.pages.mnielsen.k8s-ft.win (127.0.0.1) port 80 (#0)
> HEAD /pages-demo HTTP/1.1
> Host: root.pages.mnielsen.k8s-ft.win
> User-Agent: curl/7.77.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 302 Found
HTTP/1.1 302 Found
< content-type: text/html; charset=utf-8
content-type: text/html; charset=utf-8
< location: //root.pages.mnielsen.k8s-ft.win/pages-demo/
location: //root.pages.mnielsen.k8s-ft.win/pages-demo/
< vary: Origin
vary: Origin
< date: Mon, 06 Dec 2021 20:11:39 GMT
date: Mon, 06 Dec 2021 20:11:39 GMT

<
* Connection #0 to host root.pages.mnielsen.k8s-ft.win left intact

4. Confirm source IP in Pages logs

$ kubectl -n n2270 get logs <pod name> -c gitlab-pages
...
{"content_type":"text/html; charset=utf-8","correlation_id":"01FP8NZ50HWZ69Q5R3GNPJYA5C","duration_ms":0,"host":"root.pages.mnielsen.k8s-ft.win","level":"info","method":"HEAD","msg":"access","pages_host":"root.pages.mnielsen.k8s-ft.win","pages_https":false,"pages_project_id":2,"pages_project_prefix":"/pages-demo/","pages_project_serving_type":"zip","proto":"HTTP/1.1","referrer":"","remote_addr":"172.17.0.1","remote_ip":"172.17.0.1","status":302,"system":"http","time":"2021-12-06T20:11:39Z","ttfb_ms":0,"uri":"/pages-demo","user_agent":"curl/7.77.0","written_bytes":0}
...

Notice that remote_ip is 172.17.0.1. This matches the gateway for the HAProxy container:

$ docker inspect -f '{{range .NetworkSettings.Networks}}{{.Gateway}}{{end}}' 488e
172.17.0.1

It also matches the HAproxy logs:

<134>Dec  6 20:11:53 haproxy[9]: 172.17.0.1:58164 [06/Dec/2021:20:11:53.367] test pages/s1 0/0/36/41/77 302 168 - - ---- 1/1/0/0/0 0/0 "HEAD /pages-demo HTTP/1.1"

Required

• Merge Request Title and Description are up to date, accurate, and descriptive
• MR targeting the appropriate branch
• MR has a green pipeline on GitLab.com

Expected (please provide an explanation if not completing)

• Test plan indicating conditions for success has been posted and passes
• Documentation created/updated
• Tests added
• Integration tests added to GitLab QA
• Equivalent MR/issue for omnibus-gitlab opened