feat: enable proxy protocol on ingress controller
Description
In order for the nginx ingress pod in our cluster to be able to infer the real IP of the user, we have to enable Proxy Protocol on the load balancer that's automatically set up by the ingress-nginx helm package. The proxy protocol adds original IP info to the incoming request that nginx can then read and strip off before forwarding the request further into the cluster.
The ingress-nginx helm chart let's you enable proxy protocol on the load balancer using annotations. The annotation that we have to use depends on the cluster provider (AWS uses different annotations than DO).
In order for all pods in the cluster to be able to obtain the real IP of the user, we need to do two things in addition to enabling proxy protocol on the load balancer:
- We have to enable set the
use-proxy-protocol
flag on the nginx controller so that nginx expects incoming requests to be using the proxy protocol and handles them accordingly. - We have to enable the
use-forwarded-headers
flag so that nginx sets theX-Forwarded-For
header to the original IP as received from the load balancer using the proxy protocol, which services further down in the cluster can then use to determine the real original IP of the user.
Testing instructions
The quickest way to check whether the LMS infers the real IP of the user correctly, is to tail the logs of the LMS pod and look at the IP logged inside tracking logs (the lines that include [tracking]
). Perform a few requests on your test instance and verify that tracking logs contain your IP.
./kubectl logs -f -n <your-ns> lms-<your-pod-id> | grep tracking
Currently this change is already deployed on the grove-stage-digitalocean cluster, but not on the grove-stage-aws cluster, because Keith was doing some changes on the cluster at the same time and we were overriding each other's changes, however I did test that it works correctly at some point before my changes were rolled back.
Checklist
If any of the items below is not applicable, do not remove them, but put a check in it.
-
All providers include the new feature/change -
All affected providers can provision new clusters -
Unit tests are added/updated -
Documentation is added/updated -
The TOOLS_CONTAINER_IMAGE_VERSION
in ci_vars.yml is updated -
The grove-template repository is updated