Add parent_workflow_id to Duo workflows for flow chaining

Igor Drozdovrequested to merge
630476-add-parent-workflow-id-to-duo-workflows into master

What does this MR do and why?

We want AI flows to be able to trigger other flows, with limits on how far the chain can go. This MR is the first step of that plan. It adds only schema and model changes.

Nothing writes the new column or the new enum value yet, so there is no change in behaviour. The feature flag, the depth and breadth limits, and the service changes will come in later MRs.

Changes:

  • Adds a nullable parent_workflow_id bigint column to duo_workflows_workflows.
  • Adds the index index_duo_workflows_workflows_on_parent_workflow_id. It is created concurrently.
  • Adds the foreign key fk_duo_workflows_workflows_parent_workflow_id. It points from duo_workflows_workflows to duo_workflows_workflows(id) and uses ON DELETE SET NULL. The table is small (table_size: small in db/docs), so the foreign key is added already validated.
  • Adds the check constraint check_duo_workflows_workflows_parent_workflow_id_flow: parent_workflow_id IS NULL OR trigger_source = 4. It checks only one direction, on purpose. When a parent is deleted, the foreign key sets the child's parent_workflow_id to null. The child still has trigger_source = flow, and that must stay valid. We treat such a child as the root of its chain.
  • ClickHouse: adds parent_workflow_id Nullable(Int64) to siphon_duo_workflows_workflows, because Siphon replicates this table. This includes the migration, db/click_house/main.sql, and the schema cache.
  • Updates the Ai::DuoWorkflows::Workflow model:
    • Adds flow: 4 to the trigger_source enum.
    • Adds belongs_to :parent_workflow and has_many :child_workflows.
    • Adds a validation that requires parent_workflow when trigger_source is flow. It runs on create only, because the foreign key can set the column to null later.

Follow-ups

These will be handled in later MRs:

  • Ai::DuoWorkflows::RestartWorkflowService copies trigger_source but not parent_workflow_id. Restarting a workflow that a flow triggered would fail validation. This must be fixed in the MR that first writes trigger_source: :flow.
  • The service that sets the parent must check that the parent is in the same project or namespace as the child.

Database

  • I tested the migrations up and down locally.
  • All four PostgreSQL migrations are regular migrations on a small table.

References

Screenshots or screen recordings

Not applicable. This MR changes only the backend and the database.

How to set up and validate locally

  1. Run the migrations:

    bin/rails db:migrate

  2. In bin/rails console:

    parent = Ai::DuoWorkflows::Workflow.last
child = Ai::DuoWorkflows::Workflow.create!(project: parent.project, user: parent.user, trigger_source: :flow, parent_workflow: parent)
parent.child_workflows # => includes child
Ai::DuoWorkflows::Workflow.new(project: parent.project, user: parent.user, trigger_source: :flow).valid? # => false

  3. Run the model specs:

    bin/rspec ee/spec/models/ai/duo_workflows/workflow_spec.rb

MR acceptance checklist

Evaluate this MR against the checklist at https://docs.gitlab.com/development/code_review/#acceptance-checklist.

Merge request reports