Record why a Duo Agent Platform session was force-failed

Peter Gumesonrequested to merge
624266-record-session-drop-reason into master

What does this MR do and why?

A Duo Agent Platform session (a duo_workflows_workflows row) can be moved to the failed status by seven different code paths that mean very different things: a job failure, a missing terminal report, no activity, a workload that never got created, an API-reported failure, unmet preconditions, or a start failure. All seven land on the same status and emit the same event, so nothing downstream can tell them apart. In the 28 days to 2026-08-20, only 470 of 1,316 failed Fix Pipeline runs had any stored reason. The existing free-text summary column cannot be repurposed for this because GenerateWorkflowSummaryWorker overwrites it for exactly the rows worth classifying.

This MR covers the recording layer and carries the reason onto the analytics events. It does not change any user-facing behavior. It is a draft for the agent teams to review the reason set and naming before it merges.

What it adds:

  • A nullable drop_reason text column on duo_workflows_workflows, with a check constraint on length (nil means the session was not force-failed). The table is replicated to ClickHouse via Siphon, so this MR also adds the matching Nullable(String) column to the siphon_duo_workflows_workflows table.
  • A drop_reason string on the workflow model, validated against a frozen DROP_REASONS list of seven values: job_failed, no_status_reported, no_activity, workload_not_created, api_reported, preconditions_not_met, start_failed. It is cleared when a session is retried, since the earlier reason no longer applies.
  • A drop_reason: keyword on Ai::DuoWorkflows::UpdateWorkflowStatusService, persisted on the drop transition alongside the existing summary.
  • The reason set on all seven force-fail paths: five already route through the service and now pass the keyword; the other two call a bare .drop and set workflow.drop_reason immediately before the drop (the state machine persists dirty attributes on the transition, so this records the reason without rerouting those paths or changing the events, audit records, or notes they already emit).
  • Specs covering all seven paths plus the service persistence.
  • Carries drop_reason on the agent_platform_session_dropped event as a custom property (the built-in property stays environment), and replaces the hardcoded property: "failed" on the cleanup_stuck_agent_platform_session event with the reason.
  • Specs for the event changes.

An earlier version also covered external agent sessions (external_agent_failed). Those moved to ai_governance_sessions on master, so that path no longer writes to this table and the value was dropped.

Out of scope, tracked separately: how these sessions should display (whether a no_status_reported session should still read as failed) is a product call for the owning team, and it needs a reporting-capability signal this MR does not add, so it is left to them to build on the reason recorded here; the reconcile-versus-report race in #603514 (owned by another group); and the analytics dashboard event selection in https://gitlab.com/gitlab-data/product-analytics/-/work_items/3570 (a different repository, owned by the data team, unblocked by the enriched events here).

The reason value set is provisional and open to refinement with the owning groups.

Database review is needed for the new column and migration. Local ClickHouse was not available while authoring this change, so the ClickHouse migration artifacts (main.sql, schema cache, migration marker) were written to match the existing precedent for this table; the clickhouse:check-schema CI job is the confirmation step.

Feedback requested from the agent teams

Two things to settle before merge:

  • Naming. The column is drop_reason and the seven reason values are listed above. Four values come from the work item's candidate list verbatim (job_failed, no_status_reported, no_activity, api_reported); preconditions_not_met is the work item's job_never_started renamed to match the precondition path; and workload_not_created and start_failed were added here for paths the original list did not cover. All are easy to rename before merge.
  • Presentation is deliberately not here. How a no_status_reported session should display is left to the owning team. The right discriminator is whether the session could report, not its flow type or environment, and that is not persisted today; the affected sessions also come through as environment: :web. So that call, and the reporting-capability work it depends on, belongs with the team on #603514, built on top of the reason this MR records.

References

Screenshots or screen recordings

No UI change in this MR (recording and events only), so no screenshots.

Before After

How to set up and validate locally

  1. Trigger or simulate a flow that force-fails a session, for example one whose workload never reports a terminal status (no_status_reported).
  2. In the rails console, check Ai::DuoWorkflows::Workflow.last.drop_reason and confirm it is populated with the expected value.
  3. Repeat for other force-fail paths as needed to confirm each records its matching reason.
  4. No UI surfaces drop_reason yet; how these sessions should display is left to the owning team (see #603514).

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist.

Edited by Peter Gumeson

Merge request reports