Multiple 503 errors when retrieving chart information

Summary

When loading the staging dashboard we see many 503 errors returned by prometheus

https://staging.gitlab.com/gitlab-org/monitor/monitor-sandbox/-/environments/180571/metrics

Steps to reproduce

1. Visit the dashboard at https://staging.gitlab.com/gitlab-org/monitor/monitor-sandbox/-/environments/180571/metrics

2. Set the dashboard refresh rate to 10s.

3. Check the console error logs.

What is the current bug behavior?

After a few refreshes, multiple requests start failing with a 503 error.

The errors contain the following message:

{status: "error", message: "No prometheus server found"}

What is the expected correct behavior?

No errors are shown.

Possible fixes