[backport-1.4]: MR "Disable Rancher prometheus-adapter" needs a 1.4.x backport

MR !5401 (merged) needs to be backported to 1.4.x