Distribute scheduled pipelines from Scan Execution Policy

What does this MR do and why?

This MR updates the OrchestrationPolicyRuleScheduleNamespaceWorker to distribute scheduled pipelines created by the Scan Execution Policy. This is the first iteration to allow users to define batch limits in the scan_execution_policy.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

How to set up and validate locally

1. Enable the feature flag

echo "Feature.enable(:batched_scan_execution_scheduled_pipelines)" | rails c

2. Create a new group
3. Go to Secure > Policies
4. Click on New policy
5. Select Scan Execution policy
6. Change to .yaml mode
7. Copy the yaml below:

type: scan_execution_policy
name: schedule
description: ''
enabled: true
rules:
  - type: schedule
    cadence: 0 0 * * *
    branch_type: default
    timezone: Etc/UTC
actions:
  - scan: container_scanning

8. Click on Configure with a merge request
9. Merge the new MR to add the policy
10. Create three new projects under the group.
11. Update the DEFAULT_BATCH_SIZE in Security::OrchestrationPolicyRuleSchedule

===================================================================
diff --git a/ee/app/models/security/orchestration_policy_rule_schedule.rb b/ee/app/models/security/orchestration_policy_rule_schedule.rb
--- a/ee/app/models/security/orchestration_policy_rule_schedule.rb	(revision 59e91780fcab81c767bda99f83d473925efbc539)
+++ b/ee/app/models/security/orchestration_policy_rule_schedule.rb	(date 1710958953673)
@@ -33,7 +33,7 @@
       )
     end
 
-    DEFAULT_BATCH_SIZE = 50
+    DEFAULT_BATCH_SIZE = 2
     DEFAULT_BATCH_INTERVAL_UNIT = 'minute'
     DEFAULT_BATCH_INTERVAL_VALUE = 1

12. Trigger the schedule

group = Group.last
schedule = Security::OrchestrationPolicyRuleSchedule.last
schedule.update_column(:next_run_at, 1.day.ago)
Security::OrchestrationPolicyRuleScheduleNamespaceWorker.new.perform(schedule.id)

The first time the schedule runs it will create the missing security policy bots in the projects

13. Go to Manage > Members and verify the security policy bots were created

14. Trigger the schedule one more time

group = Group.last
schedule = Security::OrchestrationPolicyRuleSchedule.last
schedule.update_column(:next_run_at, 1.day.ago)
Security::OrchestrationPolicyRuleScheduleNamespaceWorker.new.perform(schedule.id)

15. Go to Admin > Monitoring > Background Job
16. Click on the scheduled tab and verify the Security::ScanExecutionPolicies::RuleScheduleWorker was enqueued. It should have four instances enqueued. The three projects we created and the security policy project. It should be a difference of 1 minute in their scheduled time: