Add dual-stack (IPv4 + IPv6) support for RKE2 Sylva clusters

Enable dual-stack (IPv4 + IPv6) support for RKE2 Sylva clusters fresh install, allowing workloads and platform components to operate over IPv6 also.

Ensure that Kubernetes core services, CNI, networking add-ons and node operating system are correctly configured to support IPv6 besides IPv4.

We are considering dual-stack support only for fresh install because RKE2 does not officially support enabling dual stack after the cluster was deployed in IPv4 only, so no upgrade path to enable dual-stack can be offered for existing clusters.

Goals:

  • Enable Kubernetes dual-stack networking with the focus on workloads: meaning pods, services can use IPv6 instead of IPv4, while maintaining an IPv4-centric cluster deployment and operational model
  • Support both IPv4-only and dual-stack with dual-stack as the default target in the long term
  • Provide documentation showing how to deploy and operate dual-stack clusters

Note: dual-stack implies that every interface (even additional networks) on nodes will have both IPv4 and IPv6 addressing, not a mix, one interface having IPv4 only and another interface IPv6 only.

Areas to cover:

  • enable IPv6 at Node OS level and be sure to correctly handle kernel IPv6 related parameters
  • Kubernetes control plane and core components: dual-stack Service CIDRs, dual-stack Pod CIDRs, kube-proxy support for dual-stack operation
  • CoreDNS Dual-Stack support
  • Calico CNI: IPv6 pools, network policies for IPv6, masquerading (IPv6 SNAT for egress)
  • MetalLB LoadBalancer services with IPv6 LoadBalancer IPs
  • Ingress Controllers support for IPv6

To-do list from a birds-eye view:

d-i-b

  • produce images with IPv6 enabled

s-c-c

  • update schema to support IPv6 for cluster networks
  • ippools and preallocations should not require updates since the syntax is the same and there is native support in metal3 ipam (but needs to be verified)
  • nmstate translator template needs updates to support IPv6
  • even though in theory calico/coredn/kube-proxy do not require any specific updates in rke2, we need to create/update helmChartConfigs if we want (and we do) to customize their behavior

sylva-library

  • network validation template needs updates if we want to cover IPv6 too

heat-operator

  • needs new security group creation or updates to allow the traffic
cluster-cidr: fd01::/48
service-cidr: fd02::/112

Openstack specifics

  • Each OpenStack network used on the cluster must have two subnets (IPv4 and IPv6) — adding a check for this would be useful.

  • Similarly to IPv4 subnets, exactly one and only one of the IPv6 subnets used on the cluster should have a default gateway set.

  • ipv6_address_mode and ipv6_ra_mode must be set to dhcpv6-stateful when creating IPv6 subnets, example for primary network:

$ openstack subnet create \
    --subnet-range 2001:cafe:42::/64 \
    --allocation-pool start=2001:cafe:42::40,end=2001:cafe:42::100 \
    --gateway 2001:cafe:42::1 \
    --network 0920cee5-9281-4d97-b9b1-24af0f9920f9 \
    --ip-version 6 \
    --ipv6-ra-mode dhcpv6-stateful \
    --ipv6-address-mode dhcpv6-stateful \
    primary_subnet_v6
  • IPv6 subnets must be at least /64 — Neutron DHCPv6 appears to overflow on larger prefixes (e.g., /56, /48).

Bare-metal specifics

All the networks used in the bare-metal cluster defined under cluster.capm3.networks need to specify also IPv6 subnets when the cluster is dual-stack, following the same ability to configure subnet range and static routes as possible in IPv4.