502 returned if Metrics are attempted to be loaded from a Prometheus server that no longer exists
We currently return a 502 if a user attempts to load a metrics page, which requests data from a no longer existent Prometheus server. I have seen this only with Prometheus servers behind Kubernetes so far, but it may also happen with a manually configured Prometheus server as well.
URL to reproduce: https://gitlab.com/joshlambert/prom-test/environments/368272/metrics
It looks like the following will happen:
- If the reactivecache is empty, we will generate a 502 after a long pause.
- If the user has requested the page recently, we will properly show the empty state, with the error banner.
We should try to fix the first load scenario, that generates the 502.