Add zoekt-gateway nginx container with optional TLS

Related to gitlab-org/gitlab#389749 (closed) . We need to add encryption and authentication to Zoekt. Zoekt is a HTTP server called from gitlab-rails. This MR introduces the encryption part with the following changes:

1. An nginx gateway in a container that proxies requests to zoekt-webserver
2. An optional configuration gateway.tls.certificate.enabled which will mount a named Certificate for nginx to use
3. An optional configuration gateway.tls.certificate.create which will create a Certificate using a named certmanager Issuer
4. Change the service to expose the gateway as well

These were extracted from this larger MR !14 (merged) which also had basicAuth. But the basicAuth was proving to be complicated so I wanted to separate the changes.

There are lots of configuration options available here but the idea is to give flexibility as we may have scenarios where we want the certificate created in the chart and may later change our mind and want the certificate created in the parent chart or maybe outside the charts altogether depending on what issuer we use.

I did not proxy requests to the indexserver for now as this is being re-written in gitlab-org/gitlab#402984 (closed) . Later on this gateway will likely proxy to the indexserver as well. I also don't want to remove the webserver service yet as this allows us to roll this out in a controlled way and delete the webserver later once we've migrated callers to the gateway.

Why not just use TLS built into Ingress?

I realise this could also be implemented in an Ingress or Service but it didn't solve our requirements for a few reasons:

1. This doesn't actually encrypt traffic all the way to the pod and therefore there is still unencrypted network traffic in the cluster
2. If we added basicAuth at an Ingress/Service level then the pods were still open to receive traffic from anywhere in the cluster. This was specifically a concern for infrasec because our NetworkPolicy is so broad in our deployments which means we basically allow anyone in the cluster to talk to any other unauthenticated service which allows escalating access more easily
3. This gateway implementation will likely become more GitLab specific over time and may end up being written in Go with custom logic. At present Zoekt is a very simple single server solution but over time we'll need to build sharding and replication into Zoekt and we'll need an abstraction layer somewhere for this and a gateway in front of the lower level Zoekt binaries could serve this purpose

How to test

How to test

1. Install cert manager in your cluster

   kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.11.0/cert-manager.yaml

2. Create a config file cabootstrap.yaml that will be used to bootstrap a CA on your cluster with the following content

   apiVersion: cert-manager.io/v1
   kind: ClusterIssuer
   metadata:
     name: selfsigned-issuer
   spec:
     selfSigned: {}
   ---
   apiVersion: cert-manager.io/v1
   kind: Certificate
   metadata:
     name: my-selfsigned-ca
     namespace: default
   spec:
     isCA: true
     commonName: my-selfsigned-ca
     secretName: root-secret
     privateKey:
       algorithm: ECDSA
       size: 256
     issuerRef:
       name: selfsigned-issuer
       kind: ClusterIssuer
       group: cert-manager.io
   ---
   apiVersion: cert-manager.io/v1
   kind: Issuer
   metadata:
     name: my-ca-issuer
     namespace: default
   spec:
     ca:
       secretName: root-secret

3. Apply this config:

   kubectl apply -f cabootstrap.yaml

4. Install the chart and have it create a certificate:

   helm install --set gateway.tls.certificate.enabled=true --set gateway.tls.certificate.create=true --set gateway.tls.certificate.issuer.name=my-ca-issuer --set gateway.tls.certificate.dnsNames='{zoekt.example.com}' gitlab-zoekt .

5. Curl the API and confirm it uses TLS:

   kubectl exec gitlab-zoekt-0 -- curl -XPOST -d '{"Q":"gitaly"}' 'https://127.0.0.1:8080/api/search' -i -k

6. View nginx logs

   kubectl logs gitlab-zoekt-0 zoekt-gateway