HTTPS support for jupyterhub deployment on k8s

Problem to solve

Jupyterhub deployment does not provide https.

Further details

(Include use cases, benefits, and/or goals)

Proposal

JupyterHub deployment from GitLab managed apps on k8s to support https out of the box.

This can be acomplished using cert manager:

There are a few parts to this:

1. Install cert-manager on the cluster

Original docs: https://cert-manager.readthedocs.io/en/latest/getting-started/2-installing.html

I installed it using Helm. Since we use mutual auth for now I had to hack together a rails script to generate a helm client key. Obviously when we implement this feature to deploy cert-manager via GitLab you won't need to do this. But for the record here is the script to be run from the rails console:

helm = Clusters::Applications::Helm.last; nil

File.open('/tmp/ca_cert.pem', 'w') { |f| f.write(helm.ca_cert) }; nil

client_cert = helm.issue_client_cert; nil

File.open('/tmp/key.pem', 'w') { |f| f.write(client_cert.key_string) }; nil
File.open('/tmp/cert.pem', 'w') { |f| f.write(client_cert.cert_string) }; nil

Now once we have our helm certs in place we can install cert-manager like so:

helm init --client-only --tiller-namespace gitlab-managed-apps
helm install stable/cert-manager --name cert-manager --namespace gitlab-managed-apps --tiller-namespace gitlab-managed-apps --tls --tls-ca-cert /tmp/ca_cert.pem --tls-cert /tmp/cert.pem --tls-key /tmp/key.pem

2. Configure the ClusterIssuer

Original docs: https://cert-manager.readthedocs.io/en/latest/reference/clusterissuers.html

kubectl create -f - <<EOF
apiVersion: certmanager.k8s.io/v1alpha1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: my-email@example.com
    privateKeySecretRef:
      name: letsencrypt-prod
    http01: {}
EOF

3. Configure the ingress-shim

Original docs:

• http://docs.cert-manager.io/en/latest/tutorials/acme/migrating-from-kube-lego.html?highlight=lego#configuring-ingress-shim-to-use-our-new-clusterissuer-by-default

• https://cert-manager.readthedocs.io/en/latest/reference/ingress-shim.html

helm upgrade cert-manager stable/cert-manager \
  --namespace gitlab-managed-apps --tiller-namespace gitlab-managed-apps \
  --tls --tls-ca-cert /tmp/ca_cert.pem --tls-cert /tmp/cert.pem --tls-key /tmp/key.pem \
  --set ingressShim.defaultIssuerName=letsencrypt-prod --set ingressShim.defaultIssuerKind=ClusterIssuer

NOTE After following all these steps the cert-manager should create a Certificate for you and this should automatically be picked up by the running ingress and your Auto DevOps deployed app should be reachable by http:// URL.

NOTE I did run into an issue where the HTTPS URL was still constantly returning default backend - 404. In the end I figured out by looking through kubectl logs deployments/cert-manager cert-manager --namespace gitlab-managed-apps -f that it was caused by the domain name being longer than 64 bytes. So the only way I was able to get a shorter domain name was by using a shorter group and project name as the domain name is generated by the Auto DevOps pipeline (see https://gitlab.com/gitlab-org/gitlab-ce/issues/49563)

What does success look like, and how can we measure that?

When jupyterhub is deployed, https is default.

Links / references

MR https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/23036/