[backport-1.6]: Flux breaks on upgrade from 1.3 if Thanos is disabled

A solution for issue #3368 (closed) needs to be backported to 1.6.x