Skip to content

[Duo Workflow]: compare actual patch with expected using LangSmith uuid

Alexander Chueshev requested to merge ac/support-multiple-goals-for-single-mr into main

What does this merge request do and why?

As discussed in https://gitlab.slack.com/archives/C07035GQ0TB/p1723212352260919?thread_ts=1723114146.056419&cid=C07035GQ0TB, we need to support having several goals for the same MR in our input dataset. This MR updates how we compare the actual patch with the expected one. We rely on the Langsmith uuid instead of the tuple (project_id, project_full_path, merge_request_iid).

How to set up and validate locally

Same as in !89 (merged).

Output example: https://smith.langchain.com/o/477de7ad-583e-47b6-a1c4-c4a0300e7aca/datasets/db675a33-ddbc-4add-b531-da9ce616afb8/compare?selectedSessions=63c1cdf5-0e74-41e4-9b4e-68f8e7e94207&baseline=undefined

Merge request checklist

  • Tests added for new functionality. If not, please raise an issue to follow up.
  • Documentation added/updated, if needed.

Merge request reports