Skip to content

VM for tunneling connections to server for GitLab QA

In order to test autodevops using GitLab QA we need to be able to create an internet accessible tunnel to the running instances.

We will require the following:

  • VM internet routable on ports 22,443
  • Running nginx with a custom config such that all requests are configured to be forwarded to a local port as HTTP requests (retaining original HOST headers) https://<port>.qa-tunnel.gitlab.info -> http://localhost:<port> for a range of ports (probably 20000-60000 unless there is a safer range to use). For security reasons nginx should not forward requests to ports outside this range as it may give access to resources on this machine to the internet.
  • DNS record for *.qa-tunnel.gitlab.info pointing to this VM
  • SSL certificate that covers *.qa-tunnel.gitlab.info on the VM and configured within nginx
  • SSH keys to be management via chef
  • Need to purchase an SSL cert for *.qa-tunnel.gitlab.info
  • Custom SSH configuration to allow gateway port binding GatewayPorts yes
  • SSH public/private key pair to use for gitlab-qa CI runner (and this user needs to exist on qa-tunnel.gitlab.info)

Clients will create tunnels through this machine like so (using 31414 as example port and some_path as example path):

ssh -R 31414:gitlab.test:80 qa-tunnel.gitlab.info

Such that when any request is made to https://31414.qa-tunnel.gitlab.info/some_path the nginx server will forward that request to it's http://localhost:31414/some_path (retaining HOST header here is important) and this in turn will forward to http://gitlab.test:80/some_path on client machine.

Or concisely:

https://31414.qa-tunnel.gitlab.info/some_path -> http://localhost:31414/some_path -> http://gitlab.test:80/some_path

Example Nginx Config

I was able to get this behaviour I wanted with the following nginx config:

http {
  server {
     listen 80;
     server_name ~^(?<sub>[2345]\d\d\d\d)\.qa-tunnel\.gitlab\.info;
     location / {
        proxy_pass http://127.0.0.1:$sub;
        proxy_set_header Host            $host;
        proxy_set_header X-Forwarded-For $remote_addr;
     }
  }
}

This demonstrates:

  • how to rewrite the subdomain to the port on localhost
  • how to only allow ports in the range 30000-59999 using the regex
  • forwarding host header which is important

It is missing:

  • HTTPS on port 443
  • Better handling of unmatched hosts (right now it returns 500 instead of 404, but probably this good enough to start with)

Obviously there may be additional things that should go into a production ready nginx server but I have just included this to demonstrate precisely what we need.

TODO:

  • create gitlab-qa-tunnel machine
  • create DNS records:
    • qa-tunnel.gitlab.info IN A 35.196.92.165
    • *.qa-tunnel.gitlab.info IN CNAME qa-tunnel.gitlab.info.
  • prepare cookbook
  • prepare chef configuration
    • create gitlab-qa-tunnel role based on base-debian role
    • prepare gitlab_users, authorization, openssh settings (access+sudo for production, ci groups; access for gitlab-qa-tunnel group; load recipes: gitlab_users::default, gitlab_sudo::default, gitlab-openssh::default)
    • load gitlab-qa-tunnel cookbook
    • prepare firewall entries (check first if needed, I'm almost sure that 22/tcp, 80/tcp and 443/tcp will be opened by default)
    • create secrets for gitlab-qa-tunnel role in ci-prd environment:
      • request wildcard SSL certificate for qa-tunnel.gitlab.info and *.qa-tunnel.gitlab.info hostnames
      • put the SSL key and SSL certificates chains into secrets
  • bootstrap the node: qa-tunnel.gitlab.info based on gitlab-qa-tunnel role
  • add node to Prometheus
    • in node job (public_hosts)
  • generate SSH key pair
    • generate keys
    • store both keys in 1password
  • configure the gitlab-qa-tunnel role for first users:

The above will allow us to create and test the machine. If the tunnel will work properly, then further users may get access after creating the account and assigning the gitlab-qa-tunnel group (both can be done in chef-repo similarly to https://dev.gitlab.org/cookbooks/chef-repo/blob/master/doc/user-administration.md#add-a-new-developer)

Edited by Tomasz Maczukin