Defer unenforceable policy evaluation with active pipeline

What does this MR do and why?

Fixes a race condition where UnenforceablePolicyRulesNotificationService prematurely unblocks fail-open approval rules.

!229073 (merged) added a pipeline_creating? guard in schedule_policy_synchronization to prevent enqueuing UnenforceablePolicyRulesNotificationWorker while the async CreatePipelineWorker hasn't finished. However, when the pipeline creation worker finishes before schedule_policy_synchronization runs, the Redis state is already cleared and as a result, the worker gets enqueued regardless and unblocks fail-open rules prematurely.

With this change, the service defers evaluation when a pipeline exists but is still running or the async pipeline creation worker got queued but hasn't created the pipeline record yet.

References

Code references

The race condition:

The premature unblock:

How to set up and validate locally

Same setup as !229073 (merged) (steps 1-5).

To trigger the race, add sleep 5 between prepare_for_mergeability and prepare_merge_request in AfterCreateService#execute. This ensures CreatePipelineWorker completes before schedule_policy_synchronization runs:

diff --git a/app/services/merge_requests/after_create_service.rb b/app/services/merge_requests/after_create_service.rb
index 83b17f03e690..fa27332a765c 100644
--- a/app/services/merge_requests/after_create_service.rb
+++ b/app/services/merge_requests/after_create_service.rb
@@ -6,6 +6,7 @@ def execute(merge_request)
       merge_request.ensure_merge_request_diff
 
       prepare_for_mergeability(merge_request)
+      sleep 5
       prepare_merge_request(merge_request)
       mark_merge_request_as_prepared(merge_request)

Reproduce the bug on master

  1. Open a new MR (add any file change)

  2. Immediately check the violation status in Rails console:

    mr = MergeRequest.last
    mr.scan_result_policy_violations.pluck(:id, :status)
    # => [[<id>, "warn"]]

Verify the fix

  1. Repeat steps 1-2 on this branch

  2. The violation should remain running:

    mr = MergeRequest.last
    mr.scan_result_policy_violations.pluck(:id, :status)
    # => [[<id>, "running"]]
  3. The MR should NOT be mergeable while the pipeline is running

  4. After the pipeline completes, the rule is evaluated normally

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Dominic Bauer

Merge request reports

Loading