Skip to content

Drop stuck waiting builds

The following discussion from !204709 (merged) should be addressed:

  • @hfyngvason started a discussion: (+3 comments)

    This will be approximately one new background job per Ci::Build. Are we able to support that extra load? Under normal operation, the primary concern will be the extra queue size (the jobs will be fast). But when the system is already under high load, retries could become more common and compound on the load.

diff --git a/app/services/ci/stuck_builds/drop_pending_service.rb b/app/services/ci/stuck_builds/drop_pending_service.rb
index dddd1cfb7816..eb2bc85cfee0 100644
--- a/app/services/ci/stuck_builds/drop_pending_service.rb
+++ b/app/services/ci/stuck_builds/drop_pending_service.rb
@@ -6,6 +6,7 @@ class DropPendingService
       include DropHelpers
 
       BUILD_PENDING_OUTDATED_TIMEOUT = 1.day
+      BUILD_PENDING_OUTDATED_WAITING_FOR_ACK_TIMEOUT = 1.hour
       BUILD_PENDING_STUCK_TIMEOUT = 1.hour
 
       def execute
@@ -16,6 +17,11 @@ def execute
           failure_reason: :stuck_or_timeout_failure
         )
 
+        drop(
+          pending_builds(BUILD_PENDING_OUTDATED_WAITING_FOR_ACK_TIMEOUT.ago).where.not(runner_id: nil),
+          failure_reason: :stuck_or_timeout_failure
+        )
+
         drop_stuck(
           pending_builds(BUILD_PENDING_STUCK_TIMEOUT.ago),
           failure_reason: :stuck_or_timeout_failure
Edited by 🤖 GitLab Bot 🤖