[backport-1.4]: Mitigate LB service disruption caused by Metallb LoadBalancerClass implementation

Issue #2671 (closed) needs to be backported to 1.4.x