Add dual-stack (IPv4 + IPv6) support for RKE2 Sylva clusters
Enable dual-stack (IPv4 + IPv6) support for RKE2 Sylva clusters fresh install, allowing workloads and platform components to operate over IPv6 also.
Ensure that Kubernetes core services, CNI, networking add-ons and node operating system are correctly configured to support IPv6 besides IPv4.
We are considering dual-stack support only for fresh install because RKE2 does not officially support enabling dual stack after the cluster was deployed in IPv4 only, so no upgrade path to enable dual-stack can be offered for existing clusters.
Goals:
- Enable Kubernetes dual-stack networking with the focus on workloads: meaning pods, services can use IPv6 instead of IPv4, while maintaining an IPv4-centric cluster deployment and operational model
- Support both IPv4-only and dual-stack with dual-stack as the default target in the long term
- Provide documentation showing how to deploy and operate dual-stack clusters
Note: dual-stack implies that every interface (even additional networks) on nodes will have both IPv4 and IPv6 addressing, not a mix, one interface having IPv4 only and another interface IPv6 only.
Areas to cover:
- enable IPv6 at Node OS level and be sure to correctly handle kernel IPv6 related parameters
- Kubernetes control plane and core components: dual-stack Service CIDRs, dual-stack Pod CIDRs, kube-proxy support for dual-stack operation
- CoreDNS Dual-Stack support
- Calico CNI: IPv6 pools, network policies for IPv6, masquerading (IPv6 SNAT for egress)
- MetalLB LoadBalancer services with IPv6 LoadBalancer IPs
- Ingress Controllers support for IPv6
To-do list from a birds-eye view:
d-i-b
- produce images with IPv6 enabled
s-c-c
- update schema to support IPv6 for cluster networks
- ippools and preallocations should not require updates since the syntax is the same and there is native support in metal3 ipam (but needs to be verified)
- nmstate translator template needs updates to support IPv6
- even though in theory calico/coredn/kube-proxy do not require any specific updates in rke2, we need to create/update helmChartConfigs if we want (and we do) to customize their behavior
sylva-library
- network validation template needs updates if we want to cover IPv6 too
heat-operator
- needs new security group creation or updates to allow the traffic
Recommended IPv6 CIDRs (to be added to the IPv4 ones):
cluster-cidr: fd01::/48
service-cidr: fd02::/112Openstack specifics
-
Each OpenStack network used on the cluster must have two subnets (IPv4 and IPv6) — adding a check for this would be useful.
-
Similarly to IPv4 subnets, exactly one and only one of the IPv6 subnets used on the cluster should have a default gateway set.
-
ipv6_address_modeandipv6_ra_modemust be set todhcpv6-statefulwhen creating IPv6 subnets, example for primary network:
$ openstack subnet create \
--subnet-range 2001:cafe:42::/64 \
--allocation-pool start=2001:cafe:42::40,end=2001:cafe:42::100 \
--gateway 2001:cafe:42::1 \
--network 0920cee5-9281-4d97-b9b1-24af0f9920f9 \
--ip-version 6 \
--ipv6-ra-mode dhcpv6-stateful \
--ipv6-address-mode dhcpv6-stateful \
primary_subnet_v6- IPv6 subnets must be at least
/64— Neutron DHCPv6 appears to overflow on larger prefixes (e.g.,/56,/48).
Bare-metal specifics
All the networks used in the bare-metal cluster defined under cluster.capm3.networks need to specify also IPv6 subnets when the cluster is dual-stack, following the same ability to configure subnet range and static routes as possible in IPv4.