Skip to content

Add basic code generation endpoint

Alexander Chueshev requested to merge ac/code-generation-endpoint-impl into main

This MR adds a basic /code/generation endpoint that calls the raw code-bison model without any pre/post-processing.

Example:

curl -X 'POST' \
  'http://0.0.0.0:5052/v2/code/generations' \
  -H 'accept: application/json' \
  -H 'Authorization: <AUTH>' \
  -H 'Content-Type: application/json' \
  -d '{
  "prompt_version": 1,
  "project_path": "string",
  "project_id": 0,
  "current_file": {
    "file_name": "file.py",
    "content_above_cursor": "# create a book class with two fields name and description",
    "content_below_cursor": "string"
  },
  "telemetry": []
}'

Response body

{
  "id": "id",
  "model": {
    "engine": "vertex-ai",
    "name": "code-bison",
    "lang": "python"
  },
  "object": "text_completion",
  "created": 1692037248,
  "choices": [
    {
      "text": "```\nclass Book:\n    def __init__(self,",
      "index": 0,
      "finish_reason": "length"
    }
  ]

Note: Code-bison doesn't support suffixes, however, we could reuse the suffix for better prompt engineering/stop criterion.

Another issue to make this endpoint public - #262 (closed)

Blocked by !315 (merged)

Closes gitlab-org/gitlab#421318 (closed)

Edited by Alexander Chueshev

Merge request reports