Skip to content

feat: add inference error rate metrics_host

Bob Van Landuyt requested to merge bvl/add-inference-error-metrics into main

feat: add inference error rate metrics_host

This adds a success label to the inference counter and histogram. Doing this will allow us to distinguish errors in the inference metrics and allow us to add an error-component to the SLI. This means we can alert for it.

By adding the success label to the duration histogram, we can ignore slow inferences if they result in a failure. This means that a failing slow inference would only count towards the error budget once, rather than once of errors and once for apdex.

Merge request reports