Call to validate_query in custom metrics form is not retried

Summary

The API call to validate a query entered in the Custom Metric form (Settings > Integrations > Prometheus > New Metric) is not retried if 202 Accepted is returned. This happens in the new metric page and edit metric page.

The 202 Accepted causes an error message to be displayed: There was an error trying to validate your query

Workaround

Delete one key in the query and type that key in again. This will call the validate_query API again.

Steps to reproduce

1. Open an existing project or create a new one.
2. Go to Settings > Integrations > Prometheus > New Metric or Settings > Integrations > Prometheus > Existing Metric
3. Keep the network tab in the developer console open.
4. Enter a value in the Query text box.
5. If the call to validate_query returns 202, the call is not retried.

You can force the controller to return 202 in order to be able to reliably reproduce this issue: https://gitlab.com/gitlab-org/gitlab-ee/blob/c8804c890ad213be6de9c3b285c30e02d228bb5d/ee/app/controllers/ee/projects/prometheus/metrics_controller.rb#L22

What is the current bug behavior?

GitLab is not validating queries consistently and sometimes shows an error for valid queries.

What is the expected correct behavior?

GitLab should validate queries consistently and only show errors if queries are invalid.

Relevant logs and/or screenshots

Possible fixes

We need to retry the validate API call in https://gitlab.com/gitlab-org/gitlab-ee/blob/c0ea77ac2dc82236f091bee213c9aff3a4ed9a6d/ee/app/assets/javascripts/custom_metrics/components/custom_metrics_form_fields.vue#L89