03-04 : Investigate Duo Chat metrics degradation on Issue/Epic task
Problem to solve
We observe a degradation of metrics on Issue/Epic related questions between 2029-Feb-29
and 2029-Mar-01
.
(source) (internal access)
Proposal
Investigate all metrics recorded on 2029-Mar-01
to understand the drop in the metrics.
Further details
Output tables from daily run
dev-ai-research-0e2f8974.duo_chat_daily_runs.chat_dataset_2_v1__similarity_score
dev-ai-research-0e2f8974.duo_chat_daily_runs.chat_dataset_2_v1__independent_llm_judge
dev-ai-research-0e2f8974.duo_chat_daily_runs.chat_dataset_2_v1__similarity_score
Input table for above runs
dev-ai-research-0e2f8974.duo_chat.chat_dataset_2_v1
Links / references
- Related changes gitlab-org/gitlab!145959 (merged)
Edited by Tan Le