Route CI starts and run context through ExecuteRunService
What does this MR do and why?
GitLab can start a GitLab Duo agent session on its own, for example when you mention @GitLabDuo or ask for an automatic code review. The session runs either in a CI job or inside GitLab through Workhorse. Today Rails has several separate code paths that each start sessions in their own way, so a new runtime or feature has to be built once for every path. This MR moves code review, workplan and readiness score onto the single entry point Slack already uses (Ai::DuoWorkflows::ExecuteRunService). It also defines what that entry point accepts, so the remaining paths can move onto it next.
Users should see no change, apart from the two fixes listed below.
What changes
- One entry point for these three flows.
CreateAndStartWorkflowServicenow callsExecuteRunServicewithruntime: :ci, notRuntimes::Cidirectly. - The CI runtime reads session facts from the session. Flow config and report artifacts are read from the workflow, not passed in by each caller:
- Custom AI Catalog flows use the item version the session started with.
- Other flows still use
FoundationalFlowStartParamsResolver. report_artifactscomes from the flow's registry entry, which replaces theextra_paramschannel.
- Per-turn context goes in once and reaches both runtimes.
ExecuteRunServicetakescontext:with these keys:additional_context,triggering_conversation,event_type,client_context.- On CI they go to
StartWorkflowService.client_contextis now forwarded toBuildAdditionalContextServiceas well. - On Workhorse they go through the existing
optionshash ofServerSideTurnWorker, converted to plain JSON (as_json). - Unknown keys raise an
ArgumentError.
- On CI they go to
Behaviour changes
- Fix: a resumed CI session now gets its next answer delivered to the surface that started it (Slack,
@GitLabDuonote). Before this MR, the CI path never cleared the per-turn delivery claim, so that answer was skipped as already delivered. The claim is cleared only after the resume is enqueued, so a failed resume never reopens the previous turn. - Fix: resuming a custom catalog flow on CI now sends its
flow_configand binds the workflow token to it. Before, resumes sent no flow config for these flows. - A CI start now includes a client-context envelope when the caller passes
client_context.
Details for deeper review
- Sidekiq compatibility. The
ServerSideTurnWorker#performsignature is unchanged; only new keys go into the existingoptionshash. That makes mixed deploys safe in both directions:- Old workers read only
options['client_context']and ignore the new keys. - New workers keep only the known keys of
ExecuteRunService::CONTEXT_KEYSfrom the hash. - The only Workhorse caller today (Slack) sends just
client_context.
- Old workers read only
- Why the CI reset runs after the start.
- Provisioning and token creation can fail. Resetting before them could reopen the previous turn's delivery with no new turn to close it.
- Resetting after the start leaves a theoretical window where the CI turn delivers before the reset runs. A CI turn first waits for a runner and a checkout, so this does not happen in practice.
- Workhorse keeps resetting before it enqueues, because a Workhorse turn can finish within seconds.
continuation?is captured at the start of#execute. Starting a run moves the workflow out ofcreated, so a later check would misread a first run as a continuation.workflow.goalas the input event text.CreateAndStartWorkflowServiceuses the stored goal, a text column. Code review passes the MRiidas an Integer, whichRunEventwould reject.- Spec changes.
- The old spec "does not reset the messaging turn (no live CI messaging continuation yet)" now expects the reset after a successful resume. New specs check that a failed resume and a failed provisioning both leave the previous turn claimed.
- The
report_artifactscase moved fromcreate_and_start_workflow_service_spec.rbtoruntimes/ci_spec.rb. The runtime reads it from the session now, so testing it at the caller would mean stubbing the flow registry for every lookup in the example.
- Out of scope, FYI for the next step: custom catalog flows resolve to the
fullcoding environment and theciruntime, because two places look atworkflow_definitiononly and ignore the flow's owncoding_environment:ExecuteRunService#coding_environmentresolves fromworkflow_definitiononly.Workflow#last_runtimereturns:ciif the session has a workload. Otherwise it derives the runtime fromworkflow_definition, so it is:cifor every custom flow. It is derived, not recorded, until the persisted runtime column planned in #629005 exists.- No current caller is affected, because nothing starts a custom flow on Workhorse yet.
- Once
Ai::Catalog::ExecuteWorkflowServicestarts custom flows through this service, a flow that declarescoding_environment: 'none'fails in two ways. A first run is routed to CI, not Workhorse. A resume goes to CI as well, even if the session ran on Workhorse. - To fix both, pass the item-version
flow_configintoCodingEnvironment.resolve, and makelast_runtimeread a recorded runtime or the same flow config.
References
- Consolidation issue: #629005
- Previous step (
Runtimes::Ciextraction): !257019 (merged) - Runtime seam: #628435 (closed)
- Duo for Slack epic: &23521 (closed)
Screenshots or screen recordings
Not applicable: backend only, no UI change.
How to set up and validate locally
These steps exercise both runtimes through the new entry point: a CI start (code review and workplan), session-derived flow config (custom catalog flow) and per-turn context on Workhorse. The author ran tests 1, 3 and 4 on GDK.
Before you start
-
Check out the branch, then restart Rails and Sidekiq so the turn worker runs the new code:
git fetch origin duo/refactor/629005-run-inputs git checkout duo/refactor/629005-run-inputs gdk restart rails-web rails-background-jobs -
Use a project in a Duo-enabled group with the Code Review flow turned on, and a GDK CI runner.
-
Start the Rails console like this. GDK services set this variable, but a console started from your shell does not, and token creation fails without it:
CLOUD_CONNECTOR_SELF_SIGN_TOKENS=1 bin/rails console -
Paste this helper. It shows the latest session and the CI variables it started with:
def last_run wf = Ai::DuoWorkflows::Workflow.last pipeline = wf.last_workload&.pipeline vars = pipeline&.builds&.first&.yaml_variables.to_a.to_h { |v| [v[:key].to_s, v[:value]] } { id: wf.id, flow: wf.workflow_definition, status: wf.status_name, runtime: wf.last_runtime, branch: vars['DUO_WORKFLOW_SOURCE_BRANCH'], runs_as: pipeline&.user&.user_type, flow_config_id: vars['DUO_WORKFLOW_FLOW_CONFIG_ID'] } end
Test 1: Code review starts on CI through the entry point
- On an MR with a small diff, comment
@GitLabDuo review. Use the UI, not the console: the service account's identity link needs a web request. - When the review comments appear, run
pp last_run.
Expect: flow: "code_review/v1", status: :finished, runtime: :ci, branch: the MR source branch, runs_as: "service_account", flow_config_id: "code_review".
Test 2: Workplan starts on the default branch
-
In the console:
Feature.enable(:workplan) Feature.enable(:duo_workplan_async_flow) -
On a work item, select Generate to create a workplan.
-
When the plan is posted to the work item, run
pp last_run.
Expect: flow: the workplan flow, runtime: :ci, branch: the project default branch.
Test 3: A custom catalog flow's config comes from its session
-
Run any custom flow from the AI Catalog once, so a session for it exists.
-
In the console:
wf = Ai::DuoWorkflows::Workflow.joins(ai_catalog_item_version: :item) .where(ai_catalog_items: { item_type: :flow }).where.not(project_id: nil) .order(id: :desc).detect { |w| w.ai_catalog_item_version.item.custom_flow? } params = Ai::DuoWorkflows::Runtimes::Ci.new(wf).send(:flow_config_params) [params[:flow_config] == wf.ai_catalog_item_version.definition.except('yaml_definition'), params[:flow_config_schema_version]]
Expect: [true, "v1"]. On master this returns no flow config for custom flows, which is what CI resumes used to send.
Test 4: Per-turn context reaches a Workhorse run
-
In one terminal, follow the workflow service log:
gdk tail duo-workflow-service | grep "additional context envelope" -
In the console, replace
gitlab-duo/testwith any project in a Duo-enabled group:project = Project.find_by_full_path('gitlab-duo/test') flow = Ai::Catalog::FoundationalFlow.slack_assistant_v1 wf = Ai::DuoWorkflows::CreateWorkflowService.new(container: project, current_user: User.find_by(username: 'root'), params: { goal: 'In one sentence, what is this project for?', workflow_definition: flow.foundational_flow_reference, agent_privileges: flow.agent_privileges, pre_approved_agent_privileges: flow.pre_approved_agent_privileges, environment: flow.environment }).execute[:workflow] Ai::DuoWorkflows::ExecuteRunService.new(wf, event: { type: :input, text: wf.goal }, context: { event_type: :mention, client_context: 'Asked from a Slack DM' }).execute.success? -
Wait about 30 seconds, then run
wf.reload.status_nameandwf.last_runtime.
Expect:
ExecuteRunServicereturnstrue.- The session reaches
:finishedwithlast_runtime:workhorse. - The log shows no
Unknown … agent_platform_client_contextwarning, because theslack_assistantflow declares it as an input and consumes it. A category the flow does not declare shows up asUnknown additional context envelope … has been skipped, which proves Rails sent it. At the time of writing that isagent_platform_trigger_context, which onlysecurity_reviewdeclares in the AI Gateway. Using a Symbol forevent_typealso checks that the worker arguments are converted to plain JSON.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.