[CAPO] Workload Cluster allocation: rke2-ingress-nginx-controller shall not use customer ip addresses from supplementary ippool
Based on Sylva v1.1.1 version, I am using the management cluster to allocate many Workload clusters.
For the new WC, my customer requires a specific IP address to be used by his services. As CaaS admin I select one available IP addr from the OAM subnet (e.g. 172.20.143.81) and I declare it inside our DNS.
I fulfill the values.yaml file to declare this IP in a supplementary 'lbpool1'
metallb:
l2_lbs:
address_pools:
- name: ippool1
addresses:
- 172.20.143.81/32 # declared in ODIC DNS => expect Address: 172.20.143.81" added in allowed_addresses_pairs for each neutron:port
l2_options:
advertisements:
- name: l2adv-bis
advertised_pools:
- ippool1
There are two cases: [note: the '172.20.143.75' ip address is obtained as the neutron:port to expose this WC inside the CAPO heat stack]
-
if this evolution of the values.yaml file is an 'update' and the targeted cluster is already existing, then the WC is properly updated and the new ippool is added => the customer's deployment works fine
-
if this values.yaml file is used to 'create' the WC, then this customer IP (for ippool1) is 'stolen' to serve the rke2-ingress-nginx-controller
ubuntu@rfp-test-bootstrap:~/test_loadbalancer$ k get ipaddresspools.metallb.io -A
NAMESPACE NAME AUTO ASSIGN AVOID BUGGY IPS ADDRESSES
metallb-system ippool1 true false ["172.20.143.81/32"]
metallb-system lbpool true false ["172.20.143.75/32"]
ubuntu@rfp-test-bootstrap:~/test_loadbalancer$ k get svc rke2-ingress-nginx-controller -n kube-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
rke2-ingress-nginx-controller LoadBalancer 100.73.175.210 172.20.143.81,172.20.143.75 80:30464/TCP,443:31329/TCP 46m
as a consequence the customer cannot deploy his own service
ubuntu@rfp-test-bootstrap:~/test_loadbalancer$ k describe svc nginx-service-lb-443
Name: nginx-service-lb-443
External Traffic Policy: Cluster
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning AllocationFailed 25m (x2 over 25m) metallb-controller Failed to allocate IP for "nginx-service-lb-443": can't change sharing key for "nginx-service-lb-443", address also in use by kube-system/rke2-ingress-nginx-controller
/!\ my proposal: make sure that the setup of the rke2-ingress-nginx-controller only rely on the default lbpool whose IP address is obtained on the initial neutron:port