Remove spec.externalIPs for Load Balancer services using the cluster VIP
For Sylva LoadBalancer services used for ingress-nginx-controller and for CoreDNS, we configure spec.externalIPs pointing to the cluster VIP.
Specifying externalIPs for LoadBalancer services that are managed by Metallb is uncommon and not needed, since annotations like metallb.io/loadBalancerIPs actually select the loadBalancerIP used to expose that service.
Although having spec.externalIPs does not seem to have impact with kube-proxy and Calico, we should analyze why we have this setting on LoadBalancer services and remove it if is not necessary.
$ k -n k8s-gateway get svc k8s-gateway -o yaml
apiVersion: v1
kind: Service
metadata:
annotations:
metallb.io/allow-shared-ip: cluster-external-ip
metallb.io/ip-allocated-from-pool: lbpool
metallb.io/loadBalancerIPs: 192.168.20.140
prometheus.io/port: "9153"
name: k8s-gateway
namespace: k8s-gateway
spec:
externalIPs:
- 192.168.20.140
loadBalancerClass: sylva.org/metallb-class
ports:
- name: dns-udp
nodePort: 31827
port: 53
protocol: UDP
targetPort: dns-udp
type: LoadBalancer
status:
loadBalancer:
ingress:
- ip: 192.168.20.140
ipMode: VIP
$ k get svc -n kube-system rke2-ingress-nginx-controller -o yaml
apiVersion: v1
kind: Service
metadata:
annotations:
metallb.io/allow-shared-ip: cluster-external-ip
metallb.io/ip-allocated-from-pool: lbpool
metallb.io/loadBalancerIPs: 192.168.20.140
name: rke2-ingress-nginx-controller
namespace: kube-system
spec:
externalIPs:
- 192.168.20.140
loadBalancerClass: sylva.org/metallb-class
ports:
- appProtocol: http
name: http
nodePort: 31023
port: 80
protocol: TCP
targetPort: http
- appProtocol: https
name: https
nodePort: 31633
port: 443
protocol: TCP
targetPort: https
type: LoadBalancer
status:
loadBalancer:
ingress:
- ip: 192.168.20.140
ipMode: VIP
Furthermore, in certain setups, like clusters using Cilium with Cilium's eBPF kube-proxy replacement, this setting lead to the following service reachability issue: pods running on the node selected as leader for the cluster VIP cannot reach the Load Balancer services above. Removing externalIPs from the rke2-ingress-nginx-controller service fixes the reachability issue.