Adds CI pipeline for duo workflow execution

What does this MR do and why?

This merge request introduces a new feature that allows users to start a Duo Workflow execution directly from the GitLab UI. When a user enables this feature, a new CI/CD pipeline is created and the Duo Workflow executor image is used to run the workflow. The user can provide the workflow goal, workflow ID, and other necessary parameters to start the workflow. This ensures a smooth and secure workflow execution experience for the users.

You can read more about the architecture of this at https://handbook.gitlab.com/handbook/engineering/architecture/design-documents/duo_workflow/#ci-pipeline-architecture

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Workflow execution in CI pipeline
Screen Recording 2024-08-08 at 1.41.10 AM.mov

How to set up and validate locally

  1. Install and setup a gitlab runner following steps here.

  2. Register a docker runner for project following steps here

  3. Create a PAT with api/ai_workflows scope.

  4. Set cloud connector url to the local server address where you're running duo workflow service

    host.docker.internal:50052
  5. Set GITLAB base url to

    http://host.docker.internal:3000
  6. Run duo-workflow-service and GDK(with ai-gateway) in local. Ensure you have auth enabled in duo workflow service

    DUO_WORKFLOW_AUTH__ENABLED=true
  7. Create a workflow

    POST /api/v4/ai/duo_workflows/workflows
    Host: {GDK_HOST}
    Content-Type: application/json
    Authorization: Bearer {PAT}
    
    {
        "project_id": "",
        "start_workflow": true,
        "goal": "Create a shell script named hello.sh that prints hello world"
    }
  8. OR Start an already existing workflow

    POST /api/v4/ai/duo_workflows/workflows/{workflow_id}/start
    Host: {GDK_HOST}
    Content-Type: application/json
    Authorization: Bearer {PAT}
    
    {
        "goal": "Create a shell script named hello.sh that prints hello world"
    }
  9. See the new pipeline created in the project pipelines list

    http://{GDK_HOST}/{group_name}/{project_name}/-/pipelines

  10. Check pipeline job execution

Related to #472975 (closed)

Edited by Dylan Griffith

Merge request reports

Loading