Skip to content

feat: Implement model_completion SLI alert for DWS

Adds model_completion SLI alert for Duo Workflow Service. This is to alert of potential issues with model provider.

model_completion is a combination of two prometheus metrics in DWS:

  • duo_workflow_llm_response_total (number of responses from llm provider)
  • duo_workflow_model_completion_errors_total (number of errors from llm provider)

The Ratio is computed with:

\frac{\text{duo\_workflow\_model\_completion\_errors\_total}}{\text{duo\_workflow\_llm\_response\_total} + \text{duo\_workflow\_model\_completion\_errors\_total}}

Error threshold set at 95% (5% error rate is acceptable).

Relates to gitlab-org/modelops/applied-ml/code-suggestions/ai-assist#1009 (closed)

Edited by Tim Morriss

Merge request reports

Loading