Skip to content

Apply model limits before ApplyPromptTransformations

Bruno Cardoso requested to merge bc/limit-prompt into main

What does this merge request do and why?

  • Apply model limits regardless of having any transformation specified.
  • Bump coverage of eval_codebase.py 50% -> 57%

How to set up and validate locally

  poetry run python promptlib/eval_codebase.py \
      --runner DirectRunner \
      --project unreview-poc-390200e5 \
      --region us-central1 \
      --temp_location "gs://unreview-dataflow/tmp/" \
      --save_main_session \
      --input-bq-table unreview-poc-390200e5:gl_gitlab_codebase.input_raw_v1 \
      --output-bq-table "unreview-poc-390200e5:gl_gitlab_experiments.tmp" \
      --throttle-sec 0.01 \
      --num-samples-per-file 3 \
      --model code-gecko@latest \
      --language python

Merge request checklist

  • I've ran the eval_codebase.py pipeline to validate that nothing is broken.
  • Tests added for new functionality. If not, please raise an issue to follow up.
  • Documentation added/updated, if needed.
Edited by Bruno Cardoso

Merge request reports