Scheduled scan execution policies not working for a group that has projects scheduled for deletion
Description
When a group with scheduled scan execution policies configured, has delayed project deletion setting enabled and some projects in the group are marked for deletion, the scheduled policy scans does not work for other projects which are not scheduled for deletion in the group.
Let's say there is a group G that has scheduled SEP configured and it applies to all projects in the group. The group G has 5 projects: [P1, P2, P3, P4, P5]. Of these projects, P3 is scheduled to be deleted. Now when the scheduler executes on the projects, P1 and P2 would work fine, while processing P3 we get the exception which will result in the scheduled scan not working for P4 and P5
https://sentry.gitlab.net/gitlab/gitlabcom/issues/4112219/?referrer=gitlab_plugin
Full Exception
GRPC::NotFound: 5:creating object reader: GetRepoPath: not a git repository: "/var/opt/gitlab/git-data/repositories/@hashed/fe/21/fe21cf9adc8c93e50ddc61100aacce4438825a6fb92b784cb3e1f4e24cd477b9.git".
config/initializers/enumerator_next_patch.rb:9:in `block (2 levels) in <module:EnumeratorNextPatch>'
gitlab_patch_backtrace_marker { super(*args) }
lib/gitlab/gitaly_client/call.rb:43:in `block (3 levels) in instrument_stream'
value = recording_request { response.next }
lib/gitlab/gitaly_client/call.rb:55:in `recording_request'
yield
lib/gitlab/gitaly_client/call.rb:43:in `block (2 levels) in instrument_stream'
value = recording_request { response.next }
lib/gitlab/gitaly_client/call.rb:42:in `loop'
loop do
...
(122 additional frame(s) were not displayed)
Gitlab::Git::Repository::NoRepository: 5:creating object reader: GetRepoPath: not a git repository: "/var/opt/gitlab/git-data/repositories/@hashed/fe/21/fe21cf9adc8c93e50ddc61100aacce4438825a6fb92b784cb3e1f4e24cd477b9.git".
lib/gitlab/git/wraps_gitaly_errors.rb:9:in `rescue in wrapped_gitaly_errors'
raise Gitlab::Git::Repository::NoRepository, e
lib/gitlab/git/wraps_gitaly_errors.rb:6:in `wrapped_gitaly_errors'
def wrapped_gitaly_errors(&block)
lib/gitlab/git/repository.rb:158:in `local_branches'
wrapped_gitaly_errors do
app/models/repository.rb:795:in `local_branches'
@local_branches ||= raw_repository.local_branches
app/workers/concerns/worker_context.rb:63:in `block in with_context'
Gitlab::ApplicationContext.new(**context).use { yield(**context) }
...
(107 additional frame(s) were not displayed)
5:creating object reader: GetRepoPath: not a git repository: "/var/opt/gitlab/git-data/repositories/@hashed/fe/21/fe21cf9adc8c93e50ddc61100aacce4438825a6fb92b784cb3e1f4e24cd477b9.git".
Logs: Kibana(internal)
Implementation plan
-
backend Create a new worker that wraps Security::SecurityOrchestrationPolicies::RuleScheduleService
with retry and call that fromSecurity::OrchestrationPolicyRuleScheduleWorker
and `Security::OrchestrationPolicyRuleScheduleNamespaceWorker -
backend Update Security::OrchestrationPolicyRuleScheduleWorker
andSecurity::OrchestrationPolicyRuleScheduleNamespaceWorker
to check if the project is not marked for deletion:
diff --git a/ee/app/workers/security/orchestration_policy_rule_schedule_namespace_worker.rb b/ee/app/workers/security/orchestration_policy_rule_schedule_namespace_worker.rb
index b264be9893e4..990dfb3d7fa2 100644
--- a/ee/app/workers/security/orchestration_policy_rule_schedule_namespace_worker.rb
+++ b/ee/app/workers/security/orchestration_policy_rule_schedule_namespace_worker.rb
@@ -20,7 +20,7 @@ def perform(rule_schedule_id)
schedule.schedule_next_run!
- security_orchestration_policy_configuration.namespace.all_projects.find_in_batches.each do |projects|
+ security_orchestration_policy_configuration.namespace.all_projects.not_aimed_for_deletion.find_in_batches.each do |projects|
projects.each do |project|
with_context(project: project, user: schedule.owner) do
Security::SecurityOrchestrationPolicies::RuleScheduleService
diff --git a/ee/app/workers/security/orchestration_policy_rule_schedule_worker.rb b/ee/app/workers/security/orchestration_policy_rule_schedule_worker.rb
index a5567eb7208d..627fdc4ae4a4 100644
--- a/ee/app/workers/security/orchestration_policy_rule_schedule_worker.rb
+++ b/ee/app/workers/security/orchestration_policy_rule_schedule_worker.rb
@@ -17,7 +17,7 @@ def perform
schedules.each do |schedule|
with_context(project: schedule.security_orchestration_policy_configuration.project, user: schedule.owner) do
if schedule.security_orchestration_policy_configuration.project?
- schedule_rules(schedule)
+ schedule_rules(schedule) unless schedule.security_orchestration_policy_configuration.project.marked_for_deletion?
else
Security::OrchestrationPolicyRuleScheduleNamespaceWorker.perform_async(schedule.id)
end
-
backend Update Security::OrchestrationPolicyRuleSchedule#applicable_branches
to guard access to branches
diff --git a/ee/app/models/security/orchestration_policy_rule_schedule.rb b/ee/app/models/security/orchestration_policy_rule_schedule.rb
index 86cf071fd0cc..b06b5ca78f3e 100644
--- a/ee/app/models/security/orchestration_policy_rule_schedule.rb
+++ b/ee/app/models/security/orchestration_policy_rule_schedule.rb
@@ -40,7 +40,7 @@ def applicable_branches(project = security_orchestration_policy_configuration.pr
configured_branches = policy&.dig(:rules, rule_index, :branches)
return [] if configured_branches.blank? || project.blank?
- branch_names = project.repository.branches
+ branch_names = project.repository&.branches
configured_branches
.flat_map { |pattern| RefMatcher.new(pattern).matching(branch_names).map(&:name) }