Deploy traefik for ingress
Deploys traefik for cluster ingress and routing. Uses the traefik CRD definitions for better integration with Kubernetes. This makes it not exactly the same as a traditional Kubernetes IngressController.
This also modifies how certificates are managed - before now we had a one-step bootstrap process which involved only passing a single tla-code to the jsonnet invocation, however this change fully decouples the certificate provisioning from the provisioning of the rest of the cluster.
This will give us a few benefits over nginx:
- it's fully free (no PLUS version) [Edit: This is inaccurate, there is an EE version]
- it has first class gRPC support (not some extra plugin)
- it has very good Kubernetes integration
- it logs in a nicer way for Loki.
This should supersede !153 (closed), as it's a small bit of padding on top to add TLS support.
This MR also includes a change to serve grafana from the root of the stats domain, rather than at a sub path of /grafana
. This will resolve some bugs we've experienced with this, and simplify how we access grafana (certainly my first port of call for monitoring).
Closes #128 (closed), #181 (closed), #137 (closed)