Ensure continued tracking of Code Completions
Problem to solve
When we allow direct connections from client to AI gateway, we will bypass the Rails monolith.
This causes two tacking issues:
- AI metric displayed in value stream dashboard is measured on Rails side (discussed in thread below):
- snowplow event tracking: Because the Rails monolith sends usage tracking events into Snowplow for gitlab.com (and rolled into weekly metrics for self-managed) for the
/code_suggestions/completions
endpoint,metric data and dashboards querying this data may be affected. Update: this event is not used so none of existing snowplow event metric dashboards will be affected (#428 (comment 1909388575))
This endpoint (despite its name) currently serves two use cases:
- Code Completions (this epic)
- Code Generations
Code Generation is not affected because it continues to go through Rails.
Proposal
Make sure we do not reduce fidelity of product usage tracking of Code Completions
and we track both direct and indirect requests on AI Gateway:
- To make sure that AI metric in value stream dashboard still shows valid data about AI usage, groupoptimize team will measure usage of requesting short-term user token (#428 (comment 1916420290)) and IDE clients will make sure that short-term token is refreshed only when user asks for code-completion and short-term token is missing (#428 (comment 1920126897)) to make sure that we don't request token if client is idle
- Make sure that
suggestion_requested
event on AI Gateway side is used for all code suggestion requests (IDE will use v3 endpoint, Rails currently uses v2 endpoint) and extend this event to track also if the request is direct or not by addingis_direct_request
to snowplow event context (#428 (comment 1910135687))
Further details
The event in question that is sent from gitlab-rails is code_suggestions_requested
.
It is raised here.
Links / references
Edited by Jan Provaznik