Remove spec.externalIPs for Load Balancer services using the cluster VIP

For Sylva LoadBalancer services used for ingress-nginx-controller and for CoreDNS, we configure spec.externalIPs pointing to the cluster VIP.

Specifying externalIPs for LoadBalancer services that are managed by Metallb is uncommon and not needed, since annotations like metallb.io/loadBalancerIPs actually select the loadBalancerIP used to expose that service.

Although having spec.externalIPs does not seem to have impact with kube-proxy and Calico, we should analyze why we have this setting on LoadBalancer services and remove it if is not necessary.

$ k -n k8s-gateway get svc k8s-gateway -o yaml
apiVersion: v1
kind: Service
metadata:
  annotations:
    metallb.io/allow-shared-ip: cluster-external-ip
    metallb.io/ip-allocated-from-pool: lbpool
    metallb.io/loadBalancerIPs: 192.168.20.140
    prometheus.io/port: "9153"

  name: k8s-gateway
  namespace: k8s-gateway
spec:
  externalIPs:
  - 192.168.20.140
  loadBalancerClass: sylva.org/metallb-class
  ports:
  - name: dns-udp
    nodePort: 31827
    port: 53
    protocol: UDP
    targetPort: dns-udp
  type: LoadBalancer
status:
  loadBalancer:
    ingress:
    - ip: 192.168.20.140
      ipMode: VIP

$ k get svc -n kube-system rke2-ingress-nginx-controller -o yaml
apiVersion: v1
kind: Service
metadata:
  annotations:
    metallb.io/allow-shared-ip: cluster-external-ip
    metallb.io/ip-allocated-from-pool: lbpool
    metallb.io/loadBalancerIPs: 192.168.20.140
  name: rke2-ingress-nginx-controller
  namespace: kube-system
spec:
  externalIPs:
  - 192.168.20.140
  loadBalancerClass: sylva.org/metallb-class
  ports:
  - appProtocol: http
    name: http
    nodePort: 31023
    port: 80
    protocol: TCP
    targetPort: http
  - appProtocol: https
    name: https
    nodePort: 31633
    port: 443
    protocol: TCP
    targetPort: https
  type: LoadBalancer
status:
  loadBalancer:
    ingress:
    - ip: 192.168.20.140
      ipMode: VIP

Furthermore, in certain setups, like clusters using Cilium with Cilium's eBPF kube-proxy replacement, this setting lead to the following service reachability issue: pods running on the node selected as leader for the cluster VIP cannot reach the Load Balancer services above. Removing externalIPs from the rke2-ingress-nginx-controller service fixes the reachability issue.

Assignee Loading
Time tracking Loading