Skip to content

Draft: Add PoC endpoint for unified AI Gateway architecture

Andras Herczeg requested to merge unified-ai-gateway-poc into main

This MR adds a new internal endpoint that can be used to make parameterised code suggestion requests.

The new endpoint can be called under

POST /internal/v2/code_completions

with the following payload

Attribute Type Required Description Example
prompt_version int no The version of the prompt 1
project_path string no The name of the project (max_len: 255) gitlab-orb/gitlab-shell
project_id int no The id of the project (max_len: 255) 33191677
current_file.file_name string yes The name of the current file (max_len: 255) README.md
current_file.content_above_cursor string yes The content above cursor (max_len: 100,000) import numpy as np
current_file.content_below_cursor string yes The content below cursor (max_len: 100,000) def __main__:\n
model string no The desired model provider (text-bison/code-bison/code-gecko) code-gecko
parameters.temperature float no 0.2
parameters.max_decode_steps int no 16
parameters.top_p float no 0.95
parameters.top_k int no 40
instances dict[string, string] no {"prefix": "some prefix for the model"}

Current limitations:

  • maxOutputTokens not implemented
  • Parameters and instances only sent to 3rd party models
  • Some edge cases are not covered
  • Quick and dirty fix

Relates to #163 (closed)

Edited by Andras Herczeg

Merge request reports