Add idempotency/locking for SPEP test run pipeline association
What does this MR do and why?
Addresses the non-atomic write issue between CI database (pipelines) and main database (test runs) when creating SPEP test run pipelines.
Related to #591425 (closed)
Problem
When creating a test run, the system writes to two separate databases without atomic transactions:
- CI database — pipeline record
- Main database — test run record with pipeline association
This can result in:
- Orphaned pipelines if association fails after pipeline creation
- Duplicate pipelines from worker retries
- Test runs stuck in
pendingstate
Solution
Add a new creating state and use an atomic state transition to claim the test run before pipeline creation:
UPDATE ... SET state = 'creating' WHERE id = ? AND state = 'pending'This ensures only one worker can proceed with pipeline creation. If the atomic update returns 0 rows, another worker already claimed it.
Changes:
- Add
creatingstate toScheduledPipelineExecutionPolicyTestRunmodel - Add
claim_for_pipeline_creationmethod for atomic claiming - Add
mark_as_failed!method (reused inTestRunCompletionWorker) - Update worker to use atomic state transition instead of exclusive lease locking
If pipeline creation fails after claiming, the test run is marked as failed with an error message indicating an orphaned pipeline may exist. The user can then start a new test run.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.