Skip to content

Use structured prompt for all model call

Hongtao Yang requested to merge hyang/use-structured-prompt into main

What does this merge request do and why?

This MR uses a structured prompt instead of just a simple string during LLM calls. This change is necessary to fully utilize Anthropic's message API (also OpenAI's API)

  • The components of the structured prompt is read from different templates
  • Move prompt template input tuple out of prompt-engine sub module. This is much more readable.
  • Apply the components of the structured prompt in the correct place for anthropic and openai models.

Ref: #180 #190

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

Merge request checklist

  • I've ran the affected pipeline(s) to validate that nothing is broken.
  • Tests added for new functionality. If not, please raise an issue to follow up.
  • Documentation added/updated, if needed.
Edited by Hongtao Yang

Merge request reports