[CAPO] Workload Cluster allocation: rke2-ingress-nginx-controller shall not use customer ip addresses from supplementary ippool

Based on Sylva v1.1.1 version, I am using the management cluster to allocate many Workload clusters.

For the new WC, my customer requires a specific IP address to be used by his services. As CaaS admin I select one available IP addr from the OAM subnet (e.g. 172.20.143.81) and I declare it inside our DNS.

I fulfill the values.yaml file to declare this IP in a supplementary 'lbpool1'

  metallb:
    l2_lbs:
      address_pools:
        - name: ippool1
          addresses:
            - 172.20.143.81/32 # declared in ODIC DNS => expect Address: 172.20.143.81"   added in allowed_addresses_pairs for each neutron:port
      l2_options:
        advertisements:
          - name: l2adv-bis
            advertised_pools:
              - ippool1

There are two cases: [note: the '172.20.143.75' ip address is obtained as the neutron:port to expose this WC inside the CAPO heat stack]

  1. if this evolution of the values.yaml file is an 'update' and the targeted cluster is already existing, then the WC is properly updated and the new ippool is added => the customer's deployment works fine

  2. if this values.yaml file is used to 'create' the WC, then this customer IP (for ippool1) is 'stolen' to serve the rke2-ingress-nginx-controller

ubuntu@rfp-test-bootstrap:~/test_loadbalancer$ k get ipaddresspools.metallb.io -A
NAMESPACE        NAME      AUTO ASSIGN   AVOID BUGGY IPS   ADDRESSES
metallb-system   ippool1   true          false             ["172.20.143.81/32"]
metallb-system   lbpool    true          false             ["172.20.143.75/32"]

ubuntu@rfp-test-bootstrap:~/test_loadbalancer$ k get svc rke2-ingress-nginx-controller -n kube-system
NAME                            TYPE           CLUSTER-IP       EXTERNAL-IP                   PORT(S)                      AGE
rke2-ingress-nginx-controller   LoadBalancer   100.73.175.210   172.20.143.81,172.20.143.75   80:30464/TCP,443:31329/TCP   46m

as a consequence the customer cannot deploy his own service

ubuntu@rfp-test-bootstrap:~/test_loadbalancer$ k describe svc nginx-service-lb-443 
Name:                     nginx-service-lb-443
External Traffic Policy:  Cluster
Events:
  Type     Reason            Age                From                Message
  ----     ------            ----               ----                -------
  Warning  AllocationFailed  25m (x2 over 25m)  metallb-controller  Failed to allocate IP for "nginx-service-lb-443": can't change sharing key for "nginx-service-lb-443", address also in use by kube-system/rke2-ingress-nginx-controller

/!\ my proposal: make sure that the setup of the rke2-ingress-nginx-controller only rely on the default lbpool whose IP address is obtained on the initial neutron:port

Assignee Loading
Time tracking Loading