Skip to content

Allow same models in a pipeline by adding index to the beam step labels.

Hongtao Yang requested to merge hyang/allow-same-model-in-pipeline into main

What does this merge request do and why?

Some times we want to compare the same model but with different parameters. For examples, we may want to compare 2 duo-chat models with different url, one pointing to production, the other pointing to local GDK instance. Or we may want to compare claude-3-haiku with 2 different prompt templates.

Right now, such operations are not permitted because we'll get errors like

RuntimeError: A transform with label "Apply prompt templates claude-3-haiku" already exists in the pipeline.

This is because each model will have a Beam step label of Apply prompt templates {model_name}, and Beam requires every step to have unique labels.

In this MR, we added an index to the step label, making every step labels are always unique.

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

Merge request checklist

  • I've ran the affected pipeline(s) to validate that nothing is broken.
  • Tests added for new functionality. If not, please raise an issue to follow up.
  • Documentation added/updated, if needed.

Merge request reports