Route CI starts and run context through ExecuteRunService

What does this MR do and why?

GitLab can start a GitLab Duo agent session on its own, for example when you mention @GitLabDuo or ask for an automatic code review. The session runs either in a CI job or inside GitLab through Workhorse. Today Rails has several separate code paths that each start sessions in their own way, so a new runtime or feature has to be built once for every path. This MR moves code review, workplan and readiness score onto the single entry point Slack already uses (Ai::DuoWorkflows::ExecuteRunService). It also defines what that entry point accepts, so the remaining paths can move onto it next.

Users should see no change, apart from the two fixes listed below.

What changes

  • One entry point for these three flows. CreateAndStartWorkflowService now calls ExecuteRunService with runtime: :ci, not Runtimes::Ci directly.
  • The CI runtime reads session facts from the session. Flow config and report artifacts are read from the workflow, not passed in by each caller:
    • Custom AI Catalog flows use the item version the session started with.
    • Other flows still use FoundationalFlowStartParamsResolver.
    • report_artifacts comes from the flow's registry entry, which replaces the extra_params channel.
  • Per-turn context goes in once and reaches both runtimes. ExecuteRunService takes context: with these keys: additional_context, triggering_conversation, event_type, client_context.
    • On CI they go to StartWorkflowService. client_context is now forwarded to BuildAdditionalContextService as well.
    • On Workhorse they go through the existing options hash of ServerSideTurnWorker, converted to plain JSON (as_json).
    • Unknown keys raise an ArgumentError.

Behaviour changes

  1. Fix: a resumed CI session now gets its next answer delivered to the surface that started it (Slack, @GitLabDuo note). Before this MR, the CI path never cleared the per-turn delivery claim, so that answer was skipped as already delivered. The claim is cleared only after the resume is enqueued, so a failed resume never reopens the previous turn.
  2. Fix: resuming a custom catalog flow on CI now sends its flow_config and binds the workflow token to it. Before, resumes sent no flow config for these flows.
  3. A CI start now includes a client-context envelope when the caller passes client_context.
Details for deeper review
  • Sidekiq compatibility. The ServerSideTurnWorker#perform signature is unchanged; only new keys go into the existing options hash. That makes mixed deploys safe in both directions:
    • Old workers read only options['client_context'] and ignore the new keys.
    • New workers keep only the known keys of ExecuteRunService::CONTEXT_KEYS from the hash.
    • The only Workhorse caller today (Slack) sends just client_context.
  • Why the CI reset runs after the start.
    • Provisioning and token creation can fail. Resetting before them could reopen the previous turn's delivery with no new turn to close it.
    • Resetting after the start leaves a theoretical window where the CI turn delivers before the reset runs. A CI turn first waits for a runner and a checkout, so this does not happen in practice.
    • Workhorse keeps resetting before it enqueues, because a Workhorse turn can finish within seconds.
  • continuation? is captured at the start of #execute. Starting a run moves the workflow out of created, so a later check would misread a first run as a continuation.
  • workflow.goal as the input event text. CreateAndStartWorkflowService uses the stored goal, a text column. Code review passes the MR iid as an Integer, which RunEvent would reject.
  • Spec changes.
    • The old spec "does not reset the messaging turn (no live CI messaging continuation yet)" now expects the reset after a successful resume. New specs check that a failed resume and a failed provisioning both leave the previous turn claimed.
    • The report_artifacts case moved from create_and_start_workflow_service_spec.rb to runtimes/ci_spec.rb. The runtime reads it from the session now, so testing it at the caller would mean stubbing the flow registry for every lookup in the example.
  • Out of scope, FYI for the next step: custom catalog flows resolve to the full coding environment and the ci runtime, because two places look at workflow_definition only and ignore the flow's own coding_environment:
    • ExecuteRunService#coding_environment resolves from workflow_definition only.
    • Workflow#last_runtime returns :ci if the session has a workload. Otherwise it derives the runtime from workflow_definition, so it is :ci for every custom flow. It is derived, not recorded, until the persisted runtime column planned in #629005 exists.
    • No current caller is affected, because nothing starts a custom flow on Workhorse yet.
    • Once Ai::Catalog::ExecuteWorkflowService starts custom flows through this service, a flow that declares coding_environment: 'none' fails in two ways. A first run is routed to CI, not Workhorse. A resume goes to CI as well, even if the session ran on Workhorse.
    • To fix both, pass the item-version flow_config into CodingEnvironment.resolve, and make last_runtime read a recorded runtime or the same flow config.

References

Screenshots or screen recordings

Not applicable: backend only, no UI change.

How to set up and validate locally

These steps exercise both runtimes through the new entry point: a CI start (code review and workplan), session-derived flow config (custom catalog flow) and per-turn context on Workhorse. The author ran tests 1, 3 and 4 on GDK.

Before you start

  1. Check out the branch, then restart Rails and Sidekiq so the turn worker runs the new code:

    git fetch origin duo/refactor/629005-run-inputs
    git checkout duo/refactor/629005-run-inputs
    gdk restart rails-web rails-background-jobs
  2. Use a project in a Duo-enabled group with the Code Review flow turned on, and a GDK CI runner.

  3. Start the Rails console like this. GDK services set this variable, but a console started from your shell does not, and token creation fails without it:

    CLOUD_CONNECTOR_SELF_SIGN_TOKENS=1 bin/rails console
  4. Paste this helper. It shows the latest session and the CI variables it started with:

    def last_run
      wf = Ai::DuoWorkflows::Workflow.last
      pipeline = wf.last_workload&.pipeline
      vars = pipeline&.builds&.first&.yaml_variables.to_a.to_h { |v| [v[:key].to_s, v[:value]] }
      { id: wf.id, flow: wf.workflow_definition, status: wf.status_name, runtime: wf.last_runtime,
        branch: vars['DUO_WORKFLOW_SOURCE_BRANCH'], runs_as: pipeline&.user&.user_type,
        flow_config_id: vars['DUO_WORKFLOW_FLOW_CONFIG_ID'] }
    end

Test 1: Code review starts on CI through the entry point

  1. On an MR with a small diff, comment @GitLabDuo review. Use the UI, not the console: the service account's identity link needs a web request.
  2. When the review comments appear, run pp last_run.

Expect: flow: "code_review/v1", status: :finished, runtime: :ci, branch: the MR source branch, runs_as: "service_account", flow_config_id: "code_review".

Test 2: Workplan starts on the default branch

  1. In the console:

    Feature.enable(:workplan)
    Feature.enable(:duo_workplan_async_flow)
  2. On a work item, select Generate to create a workplan.

  3. When the plan is posted to the work item, run pp last_run.

Expect: flow: the workplan flow, runtime: :ci, branch: the project default branch.

Test 3: A custom catalog flow's config comes from its session

  1. Run any custom flow from the AI Catalog once, so a session for it exists.

  2. In the console:

    wf = Ai::DuoWorkflows::Workflow.joins(ai_catalog_item_version: :item)
      .where(ai_catalog_items: { item_type: :flow }).where.not(project_id: nil)
      .order(id: :desc).detect { |w| w.ai_catalog_item_version.item.custom_flow? }
    params = Ai::DuoWorkflows::Runtimes::Ci.new(wf).send(:flow_config_params)
    [params[:flow_config] == wf.ai_catalog_item_version.definition.except('yaml_definition'),
      params[:flow_config_schema_version]]

Expect: [true, "v1"]. On master this returns no flow config for custom flows, which is what CI resumes used to send.

Test 4: Per-turn context reaches a Workhorse run

  1. In one terminal, follow the workflow service log:

    gdk tail duo-workflow-service | grep "additional context envelope"
  2. In the console, replace gitlab-duo/test with any project in a Duo-enabled group:

    project = Project.find_by_full_path('gitlab-duo/test')
    flow = Ai::Catalog::FoundationalFlow.slack_assistant_v1
    wf = Ai::DuoWorkflows::CreateWorkflowService.new(container: project, current_user: User.find_by(username: 'root'), params: {
      goal: 'In one sentence, what is this project for?', workflow_definition: flow.foundational_flow_reference,
      agent_privileges: flow.agent_privileges, pre_approved_agent_privileges: flow.pre_approved_agent_privileges,
      environment: flow.environment
    }).execute[:workflow]
    Ai::DuoWorkflows::ExecuteRunService.new(wf, event: { type: :input, text: wf.goal },
      context: { event_type: :mention, client_context: 'Asked from a Slack DM' }).execute.success?
  3. Wait about 30 seconds, then run wf.reload.status_name and wf.last_runtime.

Expect:

  • ExecuteRunService returns true.
  • The session reaches :finished with last_runtime :workhorse.
  • The log shows no Unknown … agent_platform_client_context warning, because the slack_assistant flow declares it as an input and consumes it. A category the flow does not declare shows up as Unknown additional context envelope … has been skipped, which proves Rails sent it. At the time of writing that is agent_platform_trigger_context, which only security_review declares in the AI Gateway. Using a Symbol for event_type also checks that the worker arguments are converted to plain JSON.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Merge request reports

Loading
Loading