Skip to content

Capture and write final prompt to output BQ table

Hongtao Yang requested to merge hyang/write-final-prompt into main

What does this merge request do and why?

This MR tries to capture the final prompt sent to the model. This crucial information for debugging and improvements.

  • Modify the get_batch_completion interface to return both the response as well as the final prompt (json payload).
  • Modify all the affected places where get_batch_completion interface is used.
  • Write the final prompt to output BQ table.

Ref: #180 and https://gitlab.com/gitlab-org/modelops/ai-model-validation-and-research/ai-evaluation/prompt-library/-/work_items/190

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

Merge request checklist

  • I've ran the affected pipeline(s) to validate that nothing is broken.
  • Tests added for new functionality. If not, please raise an issue to follow up.
  • Documentation added/updated, if needed.
Edited by Hongtao Yang

Merge request reports