Skip to content

Add script to fetch latest metrics from daily run

Tan Le requested to merge fetch-duo-chat-control-metrics into main

What does this merge request do and why?

Add a new command to fetch the sample input and corresponding control metrics for Duo Chat.

How to set up and validate locally

  1. Ensure GCP environment variables are setup.
  2. Check out to this merge request's branch.
  3. Run the follow command to fetch the latest Duo Chat documentation metrics from daily run (yq can be used to parse STDOUT).
    # Export to file
    $ poetry run promptlib duo-chat fetch-sample --task documentation --data control --output data/output/doc_control.csv
    
    # Print to STDOUT - piped to `yq` for better readability
    $ poetry run promptlib duo-chat fetch-sample --task documentation --data control | yq -p csv
  4. Run the follow command to fetch the latest Duo Chat documentation input from daily run (yq can be used to parse STDOUT).
    # Export to file
    $ poetry run promptlib duo-chat fetch-sample --task documentation --data input --output data/dataset/doc_input.csv
    
    # Print to STDOUT - piped to `yq` for better readability
    $ poetry run promptlib duo-chat fetch-sample --task documentation --data input | yq -p csv

Merge request checklist

  • I've ran the affected pipeline(s) to validate that nothing is broken.
  • Tests added for new functionality. If not, please raise an issue to follow up.
  • Documentation added/updated, if needed.
Edited by Tan Le

Merge request reports