MergeRequestsController#ci_environment_status is making HTTP calls to Kubernetes

originally raised in https://gitlab.com/gitlab-org/gitlab-ce/issues/63475#note_183983111, thanks @dosuken123 !

This issue I think is taking up Unicorn workers which is not good

https://dashboards.gitlab.net/d/thYzurImk/rails-controllers?orgId=1&var-action=Projects::MergeRequestsController%23ci_environments_status.json&var-database=influxdb-01-inf-gprd&from=1560716181433&to=1560738261981&fullscreen&panelId=17

This is called from deployment.has_metrics? - which executes HTTP request to a cluster for the number of environments https://gitlab.com/gitlab-org/gitlab-ee/blob/master/app/models/clusters/applications/prometheus.rb#L81. It's sequential operation and takes a long time to finish.

Call stack:

• app/serializers/environment_status_entity.rb:14:in `block in class:EnvironmentStatusEntity'
• app/models/deployment.rb:182:in `has_metrics?'
• app/models/concerns/prometheus_adapter.rb:24:in `can_query?'
• app/models/clusters/applications/prometheus.rb:82:in `prometheus_client'

Use the following failing spec:

diff --git a/spec/controllers/projects/merge_requests_controller_spec.rb b/spec/controllers/projects/merge_requests_controller_spec.rb
index 34cbf0c8723..99637cd743a 100644
--- a/spec/controllers/projects/merge_requests_controller_spec.rb
+++ b/spec/controllers/projects/merge_requests_controller_spec.rb
@@ -835,6 +835,13 @@ def post_assign_issues
         create(:merge_request, source_project: forked, target_project: project, target_branch: 'master', head_pipeline: pipeline)
       end
 
+      it 'does not make HTTP calls' do
+        cluster = create(:cluster, :provided_by_gcp, projects: [forked])
+        create(:clusters_applications_prometheus, :installed, cluster: cluster)
+
+        get_ci_environments_status
+      end
+
       it 'links to the environment on that project' do
         get_ci_environments_status
 

There are two variants of Prometheus - PrometheusService and Clusters::Application::Prometheus. It seems, PrometheusService doesn't need to perform any network calls compared to Clusters::Application::Prometheus