Draft: Libvirt-metal emulated cluster Proxy TLS

What does this MR do and why?

In jobs/5969829942 we can see connections with kubectl from the Equinix shell executor to management cluster hosted Rancher server (trying to connect to the sample workload cluster using the Rancher-generated kubeconfig) failling with:

Testing RKE2 node annotations

$ for cp_node in $(kubectl get --kubeconfig $WORKLOAD_CLUSTER_NAME-rancher.yaml nodes -o jsonpath='{ $.items[?(@.metadata.annotations.cluster\.x-k8s\.io\/owner-kind == "RKE2ControlPlane")].metadata.name}'); do # collapsed multi-line command
E0119 17:10:13.529635   63502 memcache.go:265] couldn't get current server API group list: Get "https://rancher.172.18.0.2.nip.io/k8s/clusters/c-m-gnh5rscr/api?timeout=32s": tls: failed to verify certificate: x509: certificate is valid for ingress.local, not rancher.172.18.0.2.nip.io
E0119 17:10:13.534181   63502 memcache.go:265] couldn't get current server API group list: Get "https://rancher.172.18.0.2.nip.io/k8s/clusters/c-m-gnh5rscr/api?timeout=32s": tls: failed to verify certificate: x509: certificate is valid for ingress.local, not rancher.172.18.0.2.nip.io
:
$ for md_node in $(kubectl get --kubeconfig $WORKLOAD_CLUSTER_NAME-rancher.yaml nodes -o jsonpath='{ $.items[?(@.metadata.annotations.cluster\.x-k8s\.io\/owner-kind == "MachineSet")].metadata.name}'); do # collapsed multi-line command
E0119 17:10:13.573977   63517 memcache.go:265] couldn't get current server API group list: Get "https://rancher.172.18.0.2.nip.io/k8s/clusters/c-m-gnh5rscr/api?timeout=32s": tls: failed to verify certificate: x509: certificate is valid for ingress.local, not rancher.172.18.0.2.nip.io

This change tries to make use of wildcard certificate for bootstrap cluster's proxy Ingress exposing mgmt cluster kube-api and services/ingresses, facilitated by libvirt-metal dev branch libvirt-metal-proxy-tls.

Makes use of https://cert-manager.io/docs/usage/ingress/#supported-annotations, based on https://cert-manager.io/docs/usage/ingress/#how-it-works.

Related reference(s)

Works on top of the change proposed in !1543 (closed).

Test coverage

Edited by Bogdan-Adrian Burciu

Merge request reports

Loading