Metric to track number of deployment pipelines with at least one failure

Summary

Measure the number of auto deploy pipelines that failed at least once. The failure does not have to be "permanent". A retry could have succeeded. But we want to measure the number of deployment pipelines that had at least one failure. Even if a pipeline has multiple failures, the metric should be incremented only once.

Proposal

• Add a new class similar to TrackDeploymentStarted. The class should increment a metric called delivery_deployments_failed_atleast_once.
• Add a rake task under the metrics namespace to call the above class.
• Add a CI job that runs on failure (when: on_failure) at the end of the coordinated pipeline. The job should only be called once, even if the pipeline has multiple failures.

Acceptance Criteria

• delivery_deployments_failed_atleast_once metric
• TrackDeploymentFailed class
• Rake task to increment above metric
• CI job to call the rake task upon failure of a coordinated pipeline