Add prompt token length metrics (!304) · Merge requests · GitLab.org / ModelOps / AI Assisted (formerly Applied ML) / Code Suggestions / AI Gateway · GitLab

Alejandro Rodríguez requested to merge token-length-metrics into main Aug 03, 2023

For https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/23904

We also split the metric by component (prefix, suffix) as discussed on https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/23904#note_1477372921