Enable dual-stack (IPv4 + IPv6) support for RKE2 Sylva clusters fresh install, allowing workloads and platform components to operate over IPv6 also. Ensure that Kubernetes core services, CNI, networking add-ons and node operating system are correctly configured to support IPv6 besides IPv4. We are considering dual-stack support only for fresh install because [RKE2 does not officially support](https://documentation.suse.com/cloudnative/rke2/latest/en/networking/basic_network_options.html#_dual_stack_configuration) enabling dual stack after the cluster was deployed in IPv4 only, so no upgrade path to enable dual-stack can be offered for existing clusters. Goals: - Enable Kubernetes dual-stack networking with the focus on workloads: meaning pods, services can use IPv6 instead of IPv4, while maintaining an IPv4-centric cluster deployment and operational model - Support both IPv4-only and dual-stack with dual-stack as the default target in the long term - Provide documentation showing how to deploy and operate dual-stack clusters Note: dual-stack implies that every interface (even additional networks) on nodes will have both IPv4 and IPv6 addressing, not a mix, one interface having IPv4 only and another interface IPv6 only. Areas to cover: - enable IPv6 at Node OS level and be sure to correctly handle kernel IPv6 related parameters - Kubernetes control plane and core components: dual-stack Service CIDRs, dual-stack Pod CIDRs, kube-proxy support for dual-stack operation - CoreDNS Dual-Stack support - Calico CNI: IPv6 pools, network policies for IPv6, masquerading (IPv6 SNAT for egress) - MetalLB LoadBalancer services with IPv6 LoadBalancer IPs - Ingress Controllers support for IPv6 #### To-do list from a birds-eye view: **d-i-b** * produce images with IPv6 enabled **s-c-c** * update schema to support IPv6 for cluster networks * ippools and preallocations should not require updates since the syntax is the same and there is native support in metal3 ipam (but needs to be verified) * nmstate translator template needs updates to support IPv6 * even though in theory calico/coredn/kube-proxy do not require any specific updates in rke2, we need to create/update helmChartConfigs if we want (and we do) to customize their behavior **sylva-library** * network validation template needs updates if we want to cover IPv6 too **heat-operator** * needs new security group creation or updates to allow the traffic #### Recommended IPv6 CIDRs (to be added to the IPv4 ones): ```yaml cluster-cidr: fd01::/48 service-cidr: fd02::/112 ``` #### Openstack specifics - Each OpenStack network used on the cluster must have two subnets (IPv4 and IPv6) — adding a check for this would be useful. - Similarly to IPv4 subnets, exactly one and only one of the IPv6 subnets used on the cluster should have a default gateway set. - `ipv6_address_mode` and `ipv6_ra_mode` must be set to `dhcpv6-stateful` when creating IPv6 subnets, example for primary network: ```shell $ openstack subnet create \ --subnet-range 2001:cafe:42::/64 \ --allocation-pool start=2001:cafe:42::40,end=2001:cafe:42::100 \ --gateway 2001:cafe:42::1 \ --network 0920cee5-9281-4d97-b9b1-24af0f9920f9 \ --ip-version 6 \ --ipv6-ra-mode dhcpv6-stateful \ --ipv6-address-mode dhcpv6-stateful \ primary_subnet_v6 ``` - IPv6 subnets must be at least **`/64`** — Neutron DHCPv6 appears to overflow on larger prefixes (e.g., `/56`, `/48`). #### Bare-metal specifics All the networks used in the bare-metal cluster defined under `cluster.capm3.networks` need to specify also IPv6 subnets when the cluster is dual-stack, following the same ability to configure subnet range and static routes as possible in IPv4.