Error Budget improvement: POST /api/:version/chat/completions
Problem
Currently, the score of groupduo chat Error Budget dashboard is less than 60.0, which means our feature is not performing well. We should identify the root cause and the fix it.
According to the Rails Requests Apdex, requests to the POST /api/:version/chat/completions
is slower than the default
threshold (< 1 sec).
Proposal
This endpoint is used for evaluations by groupai model validation group, hence this is not user-facing latency. We should set it to urgency :low
(< 5 sec) or ignoring it completely.
Edited by Shinya Maeda