Pods that use whereabouts IPAM with Multus NADs cannot be created when multus-rke2-whereabouts pods are older than one hour. Using any other IPAM or not specifying an IPAM is working. #### Reproduced in rke2-multus helm chart versions: - `v4.2.205` - `v4.2.300` - `v4.2.305` - `v4.2.311` #### Steps to reproduce 1. Deploy a cluster with Sylva `main` or `release-1.6` branches, enabling Multus unit, having some secondary node interface to be able to create Multus macvlan interfaces (alternative is to use bridge CNI instead of macvlan, if secondary node interface is not available). 2. Wait for rke2-multus-whereabouts pods to be running for more than 60 mins: ```shell $ k get pods -n kube-system | grep whereabouts multus-rke2-whereabouts-4sdvt 1/1 Running 0 65m multus-rke2-whereabouts-g2wll 1/1 Running 0 66m multus-rke2-whereabouts-ndhpl 1/1 Running 0 65m ``` 3. Apply the following manifest, adding a macvlan NetworkAttachmentDefinition to the pod. ```yaml --- apiVersion: v1 kind: Namespace metadata: name: multus-test labels: pod-security.kubernetes.io/enforce: privileged pod-security.kubernetes.io/enforce-version: latest --- apiVersion: "k8s.cni.cncf.io/v1" kind: NetworkAttachmentDefinition metadata: name: macvlan-ens4 namespace: multus-test spec: config: '{ "cniVersion": "0.3.1", "type": "macvlan", "mode": "bridge", "master": "ens4", "ipam": { "type": "whereabouts", "range": "10.200.0.0/24" } }' --- apiVersion: v1 kind: Pod metadata: name: pod-multus1 namespace: multus-test labels: app: macvlan iperf: server annotations: k8s.v1.cni.cncf.io/networks: | [ { "name": "macvlan-ens4", "namespace": "multus-test" } ] spec: containers: - name: multus-test-pod image: praqma/network-multitool:extra command: ["/bin/sh", "-c"] args: - | iperf3 -s & sleep infinity ports: - containerPort: 5201 securityContext: privileged: true ``` The pods will be stuck in `ContainerCreating`: ```shell root@vlado-jump:~/git_folder/sylva-core# k -n multus-test get pods NAME READY STATUS RESTARTS AGE pod-multus1 0/1 ContainerCreating 0 9m57s pod-multus2 0/1 ContainerCreating 0 9m57s pod-multus3 0/1 ContainerCreating 0 9m57s root@vlado-jump:~/git_folder/sylva-core# k -n multus-test describe pod pod-multus1 Name: pod-multus1 Namespace: multus-test Priority: 0 Service Account: default Node: wc1-rke2-capo-vlado-cp-6959d62863-68dsj/172.20.219.216 Start Time: Fri, 06 Mar 2026 15:14:14 +0000 Labels: app=macvlan iperf=server Annotations: k8s.v1.cni.cncf.io/networks: [ { "name": "macvlan-ens4", "namespace": "multus-test" } ] Status: Pending IP: IPs: <none> Containers: multus-test-pod: Container ID: Image: praqma/network-multitool:extra Image ID: Port: 5201/TCP Host Port: 0/TCP Command: /bin/sh -c Args: iperf3 -s & sleep infinity State: Waiting Reason: ContainerCreating Ready: False Restart Count: 0 Environment: <none> Mounts: /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-lzllf (ro) Conditions: Type Status PodReadyToStartContainers False Initialized True Ready False ContainersReady False PodScheduled True Volumes: kube-api-access-lzllf: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 3607 ConfigMapName: kube-root-ca.crt Optional: false DownwardAPI: true QoS Class: BestEffort Node-Selectors: kubernetes.io/hostname=wc1-rke2-capo-vlado-cp-6959d62863-68dsj Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 8m33s default-scheduler Successfully assigned multus-test/pod-multus1 to wc1-rke2-capo-vlado-cp-6959d62863-68dsj Warning FailedCreatePodSandBox 8m33s kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "56ea95f14f4c96561bd86d4668f799cf8033263a3ac7e8f01d6da9b399477300": plugin type="multus" name="multus-cni-network" failed (add): Multus: [multus-test/pod-multus1/161abcdb-adcc-47ab-b729-4761cd70f20c]: error waiting for pod: Unauthorized Warning FailedCreatePodSandBox 8m20s kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "823b81fecb69a414794ba638ddb23e626b1c2e341d2582278f4b51a342146625": plugin type="multus" name="multus-cni-network" failed (add): Multus: [multus-test/pod-multus1/161abcdb-adcc-47ab-b729-4761cd70f20c]: error waiting for pod: Unauthorized Warning FailedCreatePodSandBox 8m8s kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f78b45a15d64cd01148f4e5b30bfe809acfee92a53f7b8e9c93a0cacfe9da931": plugin type="multus" name="multus-cni-network" failed (add): Multus: [multus-test/pod-multus1/161abcdb-adcc-47ab-b729-4761cd70f20c]: error waiting for pod: Unauthorized Warning FailedCreatePodSandBox 7m54s kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "054b371e013eb5c86038e6417624cf90b7f55c21f3955ec59f58c7458689cb65": plugin type="multus" name="multus-cni-network" failed (add): Multus: [multus-test/pod-multus1/161abcdb-adcc-47ab-b729-4761cd70f20c]: error waiting for pod: Unauthorized Warning FailedCreatePodSandBox 7m42s kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "874f764a89f28101bbb464ccc1ce84fc92e050146486f0a1983a08dfab7490ee": plugin type="multus" name="multus-cni-network" failed (add): Multus: [multus-test/pod-multus1/161abcdb-adcc-47ab-b729-4761cd70f20c]: error waiting for pod: Unauthorized Warning FailedCreatePodSandBox 7m28s kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "32ba1a1a143c8bccb8ee8594a2ee98e554d67c35599d5227cb3ae36b95884487": plugin type="multus" name="multus-cni-network" failed (add): Multus: [multus-test/pod-multus1/161abcdb-adcc-47ab-b729-4761cd70f20c]: error waiting for pod: Unauthorized Warning FailedCreatePodSandBox 7m13s kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f6c8591ea2ae3c43d8cfd36fb10f20a4d8f718412cdee7d3b46cf4649d510bbf": plugin type="multus" name="multus-cni-network" failed (add): Multus: [multus-test/pod-multus1/161abcdb-adcc-47ab-b729-4761cd70f20c]: error waiting for pod: Unauthorized Warning FailedCreatePodSandBox 7m kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "67b3cfc1408033521daee5c864b873ee8dd7649e02ed6a56f13caf12fc9ffed4": plugin type="multus" name="multus-cni-network" failed (add): Multus: [multus-test/pod-multus1/161abcdb-adcc-47ab-b729-4761cd70f20c]: error waiting for pod: Unauthorized Warning FailedCreatePodSandBox 6m47s kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "0f9dab6b75f2892440098014f5d3c7a5f505bbb64a16ab98755b164213f22053": plugin type="multus" name="multus-cni-network" failed (add): Multus: [multus-test/pod-multus1/161abcdb-adcc-47ab-b729-4761cd70f20c]: error waiting for pod: Unauthorized Warning FailedCreatePodSandBox 3m5s (x17 over 6m32s) kubelet (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "620ec643f65d742607a8bbdc51db7af6918baa5d7b7ca8224f8efb796c6ae3ff": plugin type="multus" name="multus-cni-network" failed (add): Multus: [multus-test/pod-multus1/161abcdb-adcc-47ab-b729-4761cd70f20c]: error waiting for pod: Unauthorized ``` #### Whereabouts pod logs (while a script that creates/deletes pods with macvlan in a loop is running). [whereabouts_logs.txt](/uploads/5faf5605215b7ada4cd005efbfb58d41/whereabouts_logs.txt) #### Multus pod logs for another platform [multus_pod_logs.txt](/uploads/bc3b8722c545c334a91b2da3ca0cc6d9/multus_pod_logs.txt)