feat: Implement model_completion SLI alert for DWS
Adds model_completion SLI alert for Duo Workflow Service. This is to alert of potential issues with model provider.
model_completion is a combination of two prometheus metrics in DWS:
-
duo_workflow_llm_response_total(number of responses from llm provider) -
duo_workflow_model_completion_errors_total(number of errors from llm provider)
The Ratio is computed with:
\frac{\text{duo\_workflow\_model\_completion\_errors\_total}}{\text{duo\_workflow\_llm\_response\_total} + \text{duo\_workflow\_model\_completion\_errors\_total}}
Error threshold set at 95% (5% error rate is acceptable).
Relates to gitlab-org/modelops/applied-ml/code-suggestions/ai-assist#1009 (closed)
Edited by Tim Morriss