many of deploy-workload-cluster jobs fail (no access to mgmt cluster)

We have many (but not all) deploy-workload-cluster jobs failing:

Cannot access cluster, 'kubectl get nodes' gives: 00:00
E1014 20:29:50.058622   75187 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp [::1]:8080: connect: connection refused
...

This happens on main (https://gitlab.com/sylva-projects/sylva-core/-/jobs/8082945130), on 1.1.1 (eg. https://gitlab.com/sylva-projects/sylva-core/-/jobs/8082631203) but also on MRs (https://gitlab.com/sylva-projects/sylva-core/-/jobs/8081971988).