[backport-1.4]: Manage expiration of kubeconfig client certificates

Issue #2509 (closed) needs to be backported to 1.4.x