Extract CI flow execution into Runtimes::Ci
What does this MR do and why?
Each Duo feature that runs a flow on CI (code review, workplan, resuming a paused session) has its own copy of the CI setup code. This MR moves that code into one class, Ai::DuoWorkflows::Runtimes::Ci, so future runtimes and entry points only need to be wired in once.
Flows run the same as before, with two exceptions:
- Tokens are now created after the session, so a token failure marks the session as failed instead of stopping it before it's created.
- Code review, workplan and readiness score now add the flow's service account to the project and link its composite identity before starting. Slack and API starts already do this. If the service account can't be added, code review posts
DCR4001("the service account needs to be verified").
This is step 0/1 of #629005 and builds on the runtime seam from #628435 (closed).
What changes
- New
Ai::DuoWorkflows::Runtimes::Cihandles everything CI needs because it runs as a service account: OAuth and workflow tokens, service account membership, identity linking, source branch default, and choosing start or resume. ExecuteRunServiceandCreateAndStartWorkflowServicecall the runtime instead of each doing this setup themselves.ExecuteRunServicehandles resumes, for example from the Slack tool approval modal.CreateAndStartWorkflowServicehandles starts from code review, workplan and readiness score. It still creates the session and drops it on failure.
- Token failures return
invalid_oauth_tokenorinvalid_duo_workflow_token, so code review posts its token error (DCR4005). This also covers token services that raise (for exampleOauthAccessToken.create!): the runtime tracks the exception and returns the same reason.
Details for code agents and deeper review
- Callers pass extra input through two separate channels:
params_overrides: an allowlist (goal,human_approval,human_message) that may replace base params.ExecuteRunServiceuses it for resume.extra_params: additive only.CreateAndStartWorkflowServiceuses it forreport_artifacts. Reserved keys (tokens, branch, identity, flow config) are dropped, so extras can't overwrite them. Additional context isn't passed here:StartWorkflowServicebuilds it from the session's foundational flow.
- Token creation used to be a check that ran before the session was created. That check only existed because create and start were one step. Now tokens are created once, inside the runtime.
- Resumes through
ExecuteRunServicealso change: a token service that raises now returns an error response instead of raising. Before this MR, the resume path had no rescue at all. :service_account_erroris now a known reason inFailureMessageResolver, so code review doesn't send it to error tracking.ServiceAccountMemberAddServicestill logs the failure details (project, service account, membership lock).- The removed "container is nil" and "container is a group" specs in
create_and_start_workflow_service_spec.rbdon't mean a behavior change.StartWorkflowServicealready rejected sessions without a project, and the runtime now rejects them before creating tokens. - Out of scope, planned as later steps of 629005:
- moving
Ai::FlowTriggers::RunServiceonto the runtime; - removing the second composite identity link (
link_from_web_requestin the runtime next tolink!(context: :permission_check)inStartWorkflowService); - separating identity from runtime.
- moving
References
- Consolidation: #629005
- Runtime seam: #628435 (closed)
- Duo for Slack epic: &23521 (closed)
How to set up and validate locally
These steps check the four ways a Duo flow reaches the CI runtime: two start paths, the token failure path, and the resume path. The specs cover the same paths in CI; this is for seeing the real flows run.
Before you start
-
GDK with Duo Agent Platform and a CI runner. Flows run as CI jobs.
-
A project in a Duo-enabled group, with the Code Review, Developer and Generate Workplan flows turned on.
-
Start the Rails console like this. GDK services set this variable, but a console you open from your shell doesn't, and the flow token request fails without it:
CLOUD_CONNECTOR_SELF_SIGN_TOKENS=1 bin/rails console -
Paste this helper into the console. The tests use it to show the latest session:
def last_run wf = Ai::DuoWorkflows::Workflow.last pipeline = wf.last_workload&.pipeline vars = pipeline&.builds&.first&.yaml_variables.to_a.to_h { |v| [v[:key].to_s, v[:value]] } { id: wf.id, flow: wf.workflow_definition, status: wf.status_name, branch: vars['DUO_WORKFLOW_SOURCE_BRANCH'], runs_as: pipeline&.user&.user_type } end
Test 1: Code review starts on the MR source branch
- Open an MR with a small diff and comment
@GitLabDuo review. - When the review comments appear, run
pp last_run.
Expect: flow: "code_review/v1", status: :finished, branch: the MR source branch, runs_as: "service_account".
Test 2: Workplan starts on the default branch
-
In the console:
Feature.enable(:workplan) Feature.enable(:duo_workplan_async_flow) -
On a work item, select Generate to create a workplan.
-
When the plan is posted to the work item, run
pp last_run.
Expect: branch: the project default branch. The workplan flow passes no branch, so the runtime uses the default.
Test 3: A token failure drops the session and posts the specific error
-
Stop the workflow service so the flow token request fails:
gdk stop duo-workflow-service -
On an MR, comment
@GitLabDuo review. -
Run
pp last_run. -
Start the service again:
gdk start duo-workflow-service
Expect: status: :failed. The MR note has error code DCR4005 ("could not obtain the required..."), not the generic "Something went wrong while requesting a review".
Test 4: Resume after a tool approval
In the product, the Slack tool approval modal resumes a paused CI session through ExecuteRunService. Without a Slack app, you can call that service from the console instead. To make the Developer flow stop for approval, make two temporary changes in gitlab-ai-gateway:
--- a/duo_workflow_service/agent_platform/v1/flows/configs/developer/3.0.0.yml
+++ b/duo_workflow_service/agent_platform/v1/flows/configs/developer/3.0.0.yml
- name: "developer_agent"
type: AgentComponent
prompt_id: "developer_agent_prompt"
+ require_tool_approval: true
--- a/duo_workflow_service/components/tools_registry.py
+++ b/duo_workflow_service/components/tools_registry.py
+ self._ask_tool_names |= {"run_command"}
self._preapproved_tool_names -= self._ask_tool_names-
Restart the workflow service:
gdk restart duo-workflow-service -
On an issue, mention the Developer flow's service account:
@duo-developer-gitlab-duo use the run_command tool to run "ls -la" in the repo root and report the output -
Wait until the session stops for approval:
wf = Ai::DuoWorkflows::Workflow.last wf.status_name # => :tool_call_approval_required -
Approve and resume. The wrapper is only needed in the console: it keeps the service account linked to your user for the call. Sidekiq jobs and web requests do this already.
Gitlab::SafeRequestStore.ensure_request_store do Ai::DuoWorkflows::ExecuteRunService .new(wf, event: { type: :approval, approved: true }, runtime: :ci) .execute end -
When the job finishes, check the result:
pp wf.reload.status_name pp wf.workloads.order(:id).map { |w| [w.id, w.pipeline_id] } pp wf.last_workload.pipeline.builds.first.options[:script].grep(/--approval/).last
Expect:
:finished.- Two workloads with different pipelines: the first run and the resume.
- A script line ending in
duo run --existing-session-id <wf.id> ... --approval true. - The issue reply shows the
ls -laoutput.
Undo the gitlab-ai-gateway changes when you're done:
git -C ../gitlab-ai-gateway checkout -- \
duo_workflow_service/components/tools_registry.py \
duo_workflow_service/agent_platform/v1/flows/configs/developer/3.0.0.yml
gdk restart duo-workflow-serviceSpecs
bin/rspec ee/spec/services/ai/duo_workflows/runtimes/ci_spec.rb \
ee/spec/services/ai/duo_workflows/execute_run_service_spec.rb \
ee/spec/services/ai/duo_workflows/create_and_start_workflow_service_spec.rb \
ee/spec/services/ai/duo_workflows/code_review/failure_message_resolver_spec.rb \
ee/spec/services/ai/duo_workflows/code_review/review_merge_request_service_spec.rbMR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.