[Prompt Engineering] Improve Code Suggestions quality by truncating suffixes intelligently
In !200 (merged), we will be sending a suffix to the code-gecko
models, but we're truncating arbitrarily up to 1024 bytes.
The VSCode Extension sends up to 10 lines after the cursor. However, https://cloud.google.com/vertex-ai/docs/generative-ai/learn/models mentions:
For code completion, suffix represents the end of a piece of meaningful programming code. The model attempts to fill in the code in between the prefix and suffix.
This is a bit related to #167 (closed). We can be smarter about deciding what to truncate:
- Use actual tokens, not bytes.
- If the suffix is long, parse the file with TreeSitter, and only include a suffix up to the end of the method/function/etc.