Forward experiments telemetry to Code Suggestions requests
Problem to solve
To understand the effectiveness of new features and improvements made to Code Suggestions, we would like to track these experiments.
Proposal
-
Consume a new
experiments
payload from/v2/completions
request{ "id": "id", "model": { "engine": "vertex-ai", "name": "code-gecko", "lang": "" }, "experiments": [ { "name": "exp_1", "variant": 0 }, { "name": "exp_2", "variant": 1 } ], "object": "text_completion", "created": 1692068253, "choices": [ { "text": "", "index": 0, "finish_reason": "length" } ] }
-
Store the payload with the existing telemetry
-
Include the payload with telemetry when calling the subsequent
/v2/completions
{ "prompt_version": 1, "project_path": "awesome_project", "project_id": 23, "current_file": { "file_name": "main.py", "content_above_cursor": "# Implement FastAPI hello route", "content_below_cursor": "" }, "telemetry": [ { "lang": "python", "experiments": [ { "name": "exp_1", "variant": 0 }, { "name": "exp_2", "variant": 1 } ], "requests": 1, "accepts": 1, "errors": 0 } ] }
Further details
Links / references
Edited by Tan Le