Fix prefix incorrectly parsed in Vertex request
What does this merge request do and why?
Fix prefix incorrectly parsed in Vertex request
We pass StructuredPrompt
to all get_batch_completions
calls but this is
presumed string
type in VertexCodeModelHandle
. This results in 400
errors and subsequent empty response from Vertex code model such as
code-gecko
.
WARNING:root:Model name: code-gecko@002. Prediction failed for prompt: ('Human: func NewFormatService(containe
r container.Container) FormatService\nfunc (m *formatService) FindAllFormats() *[]model.Format\npackage service\n\nimport (\n\t"github.com/ybkuroki/go-webapp-sample/contain
er"\n\t"github.com/ybkuroki/go-webapp-sample/model"\n)\n\n// FormatService is a service for managing master data such as format and category.\ntype FormatService interface
{\n\tFindAllFormats() *[]model.Format\n}\n\ntype formatService struct \n\nAssistant:\n', '', ''). Error: 400 Instance prefix cannot be empty..
How to set up and validate locally
To reproduce this bug, we can run a test on the code-suggestions
eval with the following config.
{
"beam_config": {
"pipeline_options": {
"runner": "DirectRunner",
"project": "dev-ai-research-0e2f8974",
"region": "us-central1",
"temp_location": "gs://prompt-library/tmp/",
"save_main_session": false
}
},
"input_bq_table": "dev-ai-research-0e2f8974.code_suggestion.input_testcases_v1",
"output_bq_table": "dev-ai-research-0e2f8974.code_suggestion_experiments.tle_test_run_go",
"throttle_sec": 0.1,
"min_length": 25,
"batch_size": 32,
"eval_setups": [
{
"llm_config": {
"name": "code-gecko@002"
},
"languages": ["go"],
"prompt_template_config": {
"templates": [
{
"name": "example_template",
"template_path": "data/prompts/code_suggestions_simple.txt.example"
}
]
},
"transformations": ["function_signatures", "imports"],
"post_transformations": [],
"include_suffix": true
}
]
}
poetry run promptlib code-suggestions eval --test-run --sample-size 1 --config-file data/config/codesuggestion_config.json
Merge request checklist
-
I've ran the affected pipeline(s) to validate that nothing is broken. -
Tests added for new functionality. If not, please raise an issue to follow up. -
Documentation added/updated, if needed.
Edited by Tan Le