Track bulk Duo workflow execution across asynchronous workers

What does this MR do and why?

This MR changes bulk Duo workflow execution to enqueue workflow workers asynchronously (perform_async) and introduces a shared WorkflowTrackable concern used by all vulnerability workflow workers.

The goal is to decouple workflow execution from the executor thread while ensuring every finding reaches a terminal state (completed, failed, or cancelled). Each worker is responsible for updating the execution state, scheduling the next batch when appropriate, and respecting execution cancellation before starting work.

Additionally, this MR:

  • executes bulk Duo workflows asynchronously;
  • introduces a shared worker concern for execution state tracking;
  • propagates the execution ID across workflow workers;
  • tracks completion and schedules subsequent batches from workers;
  • handles worker failures and retry exhaustion by marking findings and executions as failed;
  • prevents cancelled executions from starting already-enqueued work; and
  • centralizes workflow-to-worker mappings used by bulk workflow execution.

Execution flow

StartService

├─ start execution
├─ schedule first batch
└─ enqueue workflow workers
    └─ perform_async(item_id, execution_id)




Workflow worker

├─ WorkflowTrackable

├─ fetch execution

├─ execution cancelled?
│     ├─ yes → mark finding cancelled
│     └─ no  → execute workflow

├─ success
│     ├─ mark finding completed
│     └─ enqueue next batch (if any)

├─ retries exhausted
│     └─ mark finding failed and fail execution

└─ unrecoverable error (retries disabled)
      └─ mark finding failed and fail execution

This allows bulk workflow execution to progress independently of the initial executor while ensuring execution state remains consistent across asynchronous workers.

References

Screenshots or screen recordings

Before After

How to set up and validate locally

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Ugo Nnanna Okeadu

Merge request reports

Loading