Skip to content

Forward experiments telemetry to Code Suggestions requests

Problem to solve

To understand the effectiveness of new features and improvements made to Code Suggestions, we would like to track these experiments.

Proposal

  1. Consume a new experiments payload from /v2/completions request

    {
  "id": "id",
  "model": {
    "engine": "vertex-ai",
    "name": "code-gecko",
    "lang": ""
  },
  "experiments": [
    {
      "name": "exp_1",
      "variant": 0
    }, {
      "name": "exp_2",
      "variant": 1
    }
  ],
  "object": "text_completion",
  "created": 1692068253,
  "choices": [
    {
      "text": "",
      "index": 0,
      "finish_reason": "length"
    }
  ]
}

  2. Store the payload with the existing telemetry

  3. Include the payload with telemetry when calling the subsequent /v2/completions

    {
  "prompt_version": 1,
  "project_path": "awesome_project",
  "project_id": 23,
  "current_file": {
    "file_name": "main.py",
    "content_above_cursor": "# Implement FastAPI hello route",
    "content_below_cursor": ""
  },
  "telemetry": [
    {
      "lang": "python",
      "experiments": [
        {
          "name": "exp_1",
          "variant": 0
        }, {
          "name": "exp_2",
          "variant": 1
        }
      ],
      "requests": 1,
      "accepts": 1,
      "errors": 0
    }
  ]
}

Further details

Links / references

Edited by Tan Le