Skip to content

chore: only enable langsmith tracing when requested

Dylan Griffith requested to merge 80-make-langsmith-tracing-per-workflow into main

Description

The Duo Workflow feature in GitLab has the ability to collect prompts and AI responses related to running a workflow. These are sent to Langchain. We want this to be off by default but allow team members to enable this data collection for our own users. This MR is the Duo Workflow Service part of that work. In related MRs we are now passing through a metadata hash as a json encoded string via gRPC. This is used to pass through a feature flag from gitlab-org/gitlab!167571 (merged) which switches on/off Langsmith tracing.

The Duo Workflow Service is the one that is sending this data to Langchain so we need to send the setting to the Duo Workflow Service. This is accomplished by passing the data through the following programs:

  1. LSP ->(http request)-> GitLab: to create a workflow and get token and other details
  2. LSP ->(shell out)->Duo Workflow Executor: to start the workflow
  3. Duo Workflow Executor ->(grpc)-> Duo Workflow Service: to start running the workflow

This MR introduces a new generic workflowMetadata JSON payload for sending this kind of insignificant data all the way through these 4 systems without having to update all 4 of them every time we need new data. The idea will be that we'll put more fields into this JSON payload in future. We do not intend to use this new JSON field for critical data that is necessary for the application to function as this creates more challenges for backwards compatibility.

This MR is 1 of 4 MRs needed to accomplish this:

  1. GitLab gitlab-org/gitlab!167571 (merged)
  2. LSP gitlab-org/editor-extensions/gitlab-lsp!881 (merged)
  3. Executor duo-workflow-executor!54 (merged)
  4. Service !101 (merged) 👈 (you are here)

Related Issues

https://gitlab.com/gitlab-org/duo-workflow/duo-workflow-service/-/issues/80

Edited by Dylan Griffith

Merge request reports