Support prompt template with the prepended language identifier
This MR updates the prompt template by adding a language identifier to it. To get the language identifier, we follow the same approach that was used in preprocessing the data to fine-tune the codegen model in https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/model-dev/-/blob/main/data/df/preprocessing.py#L233.
Briefly, we infer the language id from the file name extension. Examples:
- f.py ->
<python>
- f.min.js ->
<js>
We do not prepend a language identifier if the filename extension was not recognized and is not in the predefined list.
How to run the server locally
- Create virtualenv and init shell:
poetry shell
- Install dependencies:
poetry install
- Update the
.env
file in the root folder with the following variables:AUTH_BYPASS_EXTERNAL=true TRITON_HOST=localhost TRITON_PORT=8080 FASTAPI_DOCS_URL=/docs FASTAPI_OPENAPI_URL=/openapi.json FASTAPI_API_PORT=5052
- Get k8s credentials to access our k8s cluster:
gcloud container clusters get-credentials ai-assist --zone us-central1-c --project unreview-poc-390200e5
- Port-forward the triton server to access it locally:
kubectl port-forward svc/model-k8s-triton -n fauxpilot 8080:8080 --address='0.0.0.0'
- Start the model-gateway server locally:
poetry run codesuggestions
- Open
http://0.0.0.0:5052/docs
in your browser and run any requests to the codegen model
Ref: https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/issues/22BLO
Edited by Alexander Chueshev