Nested Groups and Project will not delete from the top level
Summary
Nested Groups and Project will not delete from the top level
Steps to reproduce
Create a Group, and within that group create many (in this case 250) sub groups. From within those sub groups create several projects. When deleting a group, that contains groups, that contains projects the sidekiq job fails. If you try and delete a sub group with projects that is successful.
Example Project
https://gitlab.com/gitlab-org/quality/performance-data/-/tree/master can be used to create the project
What is the current bug behavior?
What is the expected correct behavior?
The group should be removed
Relevant logs and/or screenshots
Manually deleting form the rails console
GitLab: 13.5.1-ee (a4cc9d13) EEGitLab Shell: 13.11.0
PostgreSQL: 11.6,
Loading production environment (Rails 6.0.3.3)
irb(main):001:0> GroupDestroyWorker.new.perform(11, 1)
Scoped order is ignored, it's forced to be batch order.
Scoped order is ignored, it's forced to be batch order.
Scoped order is ignored, it's forced to be batch order.
Scoped order is ignored, it's forced to be batch order.
Scoped order is ignored, it's forced to be batch order.
Scoped order is ignored, it's forced to be batch order.
Scoped order is ignored, it's forced to be batch order.
Scoped order is ignored, it's forced to be batch order.
Scoped order is ignored, it's forced to be batch order.
Scoped order is ignored, it's forced to be batch order.
Traceback (most recent call last):
16: from app/services/groups/destroy_service.rb:34:in `execute'
15: from app/models/concerns/storage/legacy_namespace.rb:109:in `rm_dir'
14: from app/models/concerns/storage/legacy_namespace.rb:109:in `each'
13: from app/models/concerns/storage/legacy_namespace.rb:114:in `block in rm_dir'
12: from lib/gitlab/gitaly_client/namespace_service.rb:12:in `allow'
11: from lib/gitlab/temporarily_allow.rb:9:in `temporarily_allow'
10: from lib/gitlab/gitaly_client/namespace_service.rb:12:in `block in allow'
9: from app/models/concerns/storage/legacy_namespace.rb:115:in `block (2 levels) in rm_dir'
8: from lib/gitlab/shell.rb:170:in `mv_namespace'
7: from lib/gitlab/gitaly_client/namespace_service.rb:40:in `rename'
6: from lib/gitlab/gitaly_client/namespace_service.rb:46:in `gitaly_client_call'
5: from lib/gitlab/gitaly_client.rb:167:in `call'
4: from lib/gitlab/gitaly_client/call.rb:17:in `call'
3: from lib/gitlab/gitaly_client/call.rb:55:in `recording_request'
2: from lib/gitlab/gitaly_client/call.rb:18:in `block in call'
1: from lib/gitlab/gitaly_client.rb:177:in `execute'
GRPC::Internal (13:failed proxying to secondary: rpc error: code = InvalidArgument desc = from directory gpt/many_groups_and_projects/gpt-subgroup-122 not found. debug_error_string:{"created":"@1604500203.889677008","description":"Error received from peer ipv4:10.207.120.12:3305","file":"src/core/lib/surface/call.cc","file_line":1055,"grpc_message":"failed proxying to secondary: rpc error: code = InvalidArgument desc = from directory gpt/many_groups_and_projects/gpt-subgroup-122 not found","grpc_status":13})
Results of GitLab environment info
Running Gitlab 13.5.1 with Gitaly Cluster