Skip to content

Add Anthropic model to the pipeline

Hongtao Yang requested to merge antropic-model into main

What does this merge request do and why?

This MR adds Antropic claude v2 to the pipeline.

How to set up and validate locally

Here is the script that I used to test the pipeline locally:

poetry run python promptlib/eval_codebase.py \
    --runner DirectRunner \
    --project unreview-poc-390200e5 \
    --region us-central1 \
    --temp_location "gs://unreview-dataflow/tmp/" \
    --save_main_session \
    --input_bq_table unreview-poc-390200e5:gl_gitlab_codebase.content_v1 \
    --output-bq-table unreview-poc-390200e5:gl_gitlab_experiments.dbernardi_092123_anthropic_v1 \
    --throttle_sec 0.01 \
    --num_samples_per_file 5 \
    --model claude-2 \
    --language python \
    --anthropic-token $ANTHROPIC_API_KEY \

If you need an API key, head here to create one: Accessing Anthropic API.

The output table that proves this pipeline is working can be found here.

Merge request checklist

  • I've ran the eval_codebase.py pipeline to validate that nothing is broken.
  • Tests added for new functionality. If not, please raise an issue to follow up.
  • Documentation added/updated, if needed.
Edited by Dylan Bernardi

Merge request reports