Draft: Add PoC endpoint for unified AI Gateway architecture
This MR adds a new internal endpoint that can be used to make parameterised code suggestion requests.
The new endpoint can be called under
POST /internal/v2/code_completions
with the following payload
Attribute | Type | Required | Description | Example |
---|---|---|---|---|
prompt_version |
int | no | The version of the prompt | 1 |
project_path |
string | no | The name of the project (max_len: 255) | gitlab-orb/gitlab-shell |
project_id |
int | no | The id of the project (max_len: 255) | 33191677 |
current_file.file_name |
string | yes | The name of the current file (max_len: 255) | README.md |
current_file.content_above_cursor |
string | yes | The content above cursor (max_len: 100,000) | import numpy as np |
current_file.content_below_cursor |
string | yes | The content below cursor (max_len: 100,000) | def __main__:\n |
model |
string | no | The desired model provider (text-bison /code-bison /code-gecko ) |
code-gecko |
parameters.temperature |
float | no | 0.2 | |
parameters.max_decode_steps |
int | no | 16 | |
parameters.top_p |
float | no | 0.95 | |
parameters.top_k |
int | no | 40 | |
instances |
dict[string, string] | no | {"prefix": "some prefix for the model"} |
Current limitations:
-
maxOutputTokens
not implemented - Parameters and instances only sent to 3rd party models
- Some edge cases are not covered
- Quick and dirty fix
Relates to #163 (closed)
Edited by Andras Herczeg