[ubuntu] - kube-proxy are in crashloopbackoff due to iptables-restore error (kernel issue?)
While debugging another problem on @sakuchoudhary98's platform, we realized that his kube-proxy pod was crashloopbackoff with the following error:
E0422 21:19:09.569052 1 proxier.go:1564] "Failed to execute iptables-restore" err=<
exit status 2: Warning: Extension MARK revision 0 not supported, missing kernel module?
ip6tables-restore v1.8.9 (nf_tables): unknown option "--xor-mark"
Error occurred at line: 17
Try `ip6tables-restore -h' or 'ip6tables-restore --help' for more information.
> ipFamily="IPv6"
I0422 21:19:09.569082 1 proxier.go:833] "Sync failed" ipFamily="IPv6" retryingTime="30s"
E0422 21:19:39.610796 1 proxier.go:1564] "Failed to execute iptables-restore" err=<
exit status 2: Warning: Extension MARK revision 0 not supported, missing kernel module?
ip6tables-restore v1.8.9 (nf_tables): unknown option "--xor-mark"
Error occurred at line: 17
Try `ip6tables-restore -h' or 'ip6tables-restore --help' for more information.
> ipFamily="IPv6"
I0422 21:19:39.610824 1 proxier.go:833] "Sync failed" ipFamily="IPv6" retryingTime="30s"
E0422 21:20:09.655713 1 proxier.go:1564] "Failed to execute iptables-restore" err=<
exit status 2: Warning: Extension MARK revision 0 not supported, missing kernel module?
ip6tables-restore v1.8.9 (nf_tables): unknown option "--xor-mark"
Error occurred at line: 17
Try `ip6tables-restore -h' or 'ip6tables-restore --help' for more information.
> ipFamily="IPv6"
I0422 21:20:09.655747 1 proxier.go:833] "Sync failed" ipFamily="IPv6" retryingTime="30s"
E0422 21:20:39.703290 1 proxier.go:1564] "Failed to execute iptables-restore" err=<
exit status 2: Warning: Extension MARK revision 0 not supported, missing kernel module?
ip6tables-restore v1.8.9 (nf_tables): unknown option "--xor-mark"
Error occurred at line: 17
Try `ip6tables-restore -h' or 'ip6tables-restore --help' for more information.
> ipFamily="IPv6"
I0422 21:20:39.703317 1 proxier.go:833] "Sync failed" ipFamily="IPv6" retryingTime="30s"
E0422 21:21:09.749056 1 proxier.go:1564] "Failed to execute iptables-restore" err=<
exit status 2: Warning: Extension MARK revision 0 not supported, missing kernel module?
ip6tables-restore v1.8.9 (nf_tables): unknown option "--xor-mark"
Error occurred at line: 17
Try `ip6tables-restore -h' or 'ip6tables-restore --help' for more information.
> ipFamily="IPv6"
I0422 21:21:09.749082 1 proxier.go:833] "Sync failed" ipFamily="IPv6" retryingTime="30s"Looking more closely at the CI, we also have the same problem on capm3 runs on Ubuntu https://gitlab.com/sylva-projects/sylva-core/-/jobs/9792957905. (But the CI didn't detect the problem)
About the kubeproxy issue it seems related to the kernel used by our ubuntu image (https://github.com/k3s-io/k3s/issues/11175#issuecomment-2786404592 and https://jmcglock.substack.com/p/kube-proxy-pods-stuck-in-crashloopbackoff)