Skip to content

Add code-bison-32k model

Tan Le requested to merge add-code-bison-32k into main

What does this merge request do and why?

We need another variant of code-bison that can handle large context window such as log traces in Root Cause Analysis. code-bison can only handle up to 6,144 tokens while code-bison-32k can do up to 32,000 tokens.

https://cloud.google.com/vertex-ai/generative-ai/docs/learn/model-versioning#palm-model-versions

Relates to #327

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

Merge request checklist

  • I've ran the affected pipeline(s) to validate that nothing is broken.
  • Tests added for new functionality. If not, please raise an issue to follow up.
  • Documentation added/updated, if needed.
Edited by Tan Le

Merge request reports