Backfill ai_governance_sessions from existing Duo workflows

What does this MR do and why?

Adds a batched background migration, BackfillAiGovernanceSessionsFromDuoWorkflows, that copies every existing Duo workflow (duo_workflows_workflows) into ai_governance_sessions. A post-deploy migration queues it.

Why: the live sync added in !256733 (merged) (Ai::Governance::Session.sync_from_workflow!, called by Ai::DuoWorkflows::SyncSessionArtifactWorker) only writes a row when a workflow changes, so workflows from before that sync started have no row. This backfill fills that gap.

It uses the same field mapping as sync_from_workflow!: workflow_id, user_id, project_id, namespace_id (workflow's namespace, or the project's project_namespace_id), status, flow_type (workflow_definition), agent_type, source (gitlab_duo), session_started_at (workflow's created_at). It never sets session_finished_at; another writer owns that field.

It skips private messaging sessions (Slack adapter), just like the live sync. It also skips environment = external (glab sessions), which move separately under https://gitlab.com/gitlab-org/gitlab/-/work_items/630013. The live sync does not skip these.

It's idempotent with the live sync: inserts use ON CONFLICT ... DO NOTHING on the partial unique index i_ai_governance_sessions_on_namespace_workflow, so an existing (newer) row from the live sync wins.

Backfills all history, not just a recent window. Postgres rows are later removed once Siphon replicates them to ClickHouse; that cleanup is separate.

Query plan / estimate

About 6.44 million Duo workflows exist on GitLab.com. Batch size 5,000, sub-batch 1,000, 2 minute interval. Estimate: about 1,745 batches, roughly 58 hours (2.5 to 4 days with throttling). This adds about 2,500 rows per minute of Siphon replication load.

Database Lab numbers

• Total workflows: 6,443,514 (max id about 8.72 million)
• External (glab) workflows: 13; private messaging (Slack) rows: a few hundred

Query plan

Raw SQL for one sub-batch (batch size 5,000, sub-batch size 1,000, 2 minute interval):

WITH batch AS MATERIALIZED (
  SELECT "duo_workflows_workflows"."id" FROM "duo_workflows_workflows"
  WHERE "duo_workflows_workflows"."id" BETWEEN :start AND :end
    AND "duo_workflows_workflows"."id" >= :sub_batch_start
  LIMIT 1000
)
INSERT INTO ai_governance_sessions
  (workflow_id, user_id, project_id, namespace_id, status, flow_type, agent_type,
   source, session_started_at, created_at, updated_at)
SELECT
  w.id, w.user_id, w.project_id, COALESCE(w.namespace_id, p.project_namespace_id),
  w.status, w.workflow_definition, w.agent_type,
  0, w.created_at, NOW(), NOW()
FROM duo_workflows_workflows w
LEFT JOIN projects p ON p.id = w.project_id
WHERE w.id IN (SELECT id FROM batch)
  AND w.environment IS DISTINCT FROM 6
  AND (w.messaging_callback_context ->> 'adapter') IS DISTINCT FROM 'slack'
ON CONFLICT (namespace_id, workflow_id) WHERE workflow_id IS NOT NULL DO NOTHING;

The Database Lab version replaced the BETWEEN range with an OFFSET helper subquery to pick a realistic start ID, because Joe shows plans, not rows. The real migration uses a fixed ID range from the batching framework and does not run the helper. All lookups use primary key indexes (duo_workflows_workflows_pkey, projects_pkey), with no sequential scans.

• Cold cache: insert about 2.1 s for 998 rows, mostly disk reads. Total 12.8 s, including 10.7 s for the helper.
• Warm cache: insert about 47 ms for 998 rows. Total 176 ms, including 123 ms for the helper.

How to verify

bundle exec rspec ee/spec/lib/ee/gitlab/background_migration/backfill_ai_governance_sessions_from_duo_workflows_spec.rb spec/migrations/20260928100728_queue_backfill_ai_governance_sessions_from_duo_workflows_spec.rb

Covers the field mapping, namespace-level workflows, skip rules, that existing live-sync rows are kept, and idempotency. The EE spec runs the backfill next to the real sync_from_workflow! and confirms both write identical rows with no duplicates.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist.

Related to https://gitlab.com/gitlab-org/gitlab/-/work_items/630679